Method and apparatus for colour imaging a three-dimensional structure

ABSTRACT

A device for determining the surface topology and associated color of a structure, such as a teeth segment, includes a scanner for providing depth data for points along a two-dimensional array substantially orthogonal to the depth direction, and an image acquisition means for providing color data for each of the points of the array, while the spatial disposition of the device with respect to the structure is maintained substantially unchanged. A processor combines the color data and depth data for each point in the array, thereby providing a three-dimensional color virtual model of the surface of the structure. A corresponding method for determining the surface topology and associate color of a structure is also provided.

CROSS-REFERENCE

This application is a continuation of U.S. patent application Ser. No.16/270,419, filed Feb. 7, 2019, which is a continuation of U.S. patentapplication Ser. No. 15/175,267, filed Jun. 7, 2016, which is acontinuation of U.S. patent application Ser. No. 14/755,171, filed onJun. 30, 2015, now U.S. Pat. No. 9,404,740, issued Aug. 2, 2016, whichis a continuation of U.S. patent application Ser. No. 14/511,091, filedon Oct. 9, 2014, now U.S. Pat. No. 9,101,433, issued Aug. 11, 2015,which is a continuation of U.S. patent application Ser. No. 14/150,505,filed on Jan. 8, 2014, now U.S. Pat. No. 8,885,175, issued Nov. 11,2014, which is a continuation of U.S. patent application Ser. No.13/868,926, filed on Apr. 23, 2013, now U.S. Pat. No. 8,675,207, issuedMar. 18, 2014, which is a continuation of U.S. patent application Ser.No. 13/620,159, filed on Sep. 14, 2012, now U.S. Pat. No. 8,451,456,issued May 28, 2013, which is a continuation of U.S. patent applicationSer. No. 13/333,351, filed on Dec. 21, 2011, now U.S. Pat. No.8,363,228, issued Jan. 29, 2013, which is a continuation of U.S. patentapplication Ser. No. 12/770,379, filed on Apr. 29, 2010, now U.S. Pat.No. 8,102,538, issued Jan. 24, 2012, which is a continuation of U.S.patent application Ser. No. 12/379,343, filed on Feb. 19, 2009, now U.S.Pat. No. 7,724,378, issued May 25, 2010, which is a continuation of U.S.patent application Ser. No. 11/889,112, filed on Aug. 9, 2007, now U.S.Pat. No. 7,511,829, issued Mar. 31, 2009, which is a continuation ofU.S. patent application Ser. No. 11/154,520, filed on Jun. 17, 2005, nowU.S. Pat. No. 7,319,529, issued Jan. 15, 2008, an application claimingthe benefit under 35 U.S.C. § 119(e) of U.S. Provisional Application No.60/580,109, filed on Jun. 17, 2004, and claiming the benefit under 35U.S.C. § 119(e) of U.S. Provisional Application No. 60/580,108, filed onJun. 17, 2004, the contents of each of which are hereby incorporated byreference in their entirety.

FIELD OF THE INVENTION

The present invention relates to optical scanners, particularly forproviding a digital representation of three-dimensional objectsincluding color. The invention finds particular application in thesurveying of the intraoral cavity.

BACKGROUND OF THE INVENTION

Many methods have been developed for obtaining the three dimensionallocation of surface points of an object, for a host of applicationsincluding, inter alia, the intraoral cavity. Techniques for directnon-contact optical measurement, in particular for direct opticalmeasurement of teeth and the subsequent automatic manufacture ofdentures, are known. The term “direct optical measurement” signifiessurveying of teeth in the oral cavity of a patient. This facilitates theobtainment of digital constructional data necessary for thecomputer-assisted design (CAD) or computer-assisted manufacture (CAM) oftooth replacements without having to make any cast impressions of theteeth. Such systems typically include an optical probe coupled to anoptical pick-up or receiver such as charge coupled device (CCD) and aprocessor implementing a suitable image processing technique to designand fabricate virtually the desired product. Such methods include, forexample, confocal imaging techniques as described in WO 00/08415assigned to the present assignee. These methods provide a digitalthree-dimensional surface model that is inherently monochromatic, i.e.,no color information is obtained in the imaging process.

Associating color information with three-dimensional objects is notstraightforward, particularly when the position information is obtainedby using a three dimensional scanning method, while the colorinformation is obtained by using a two dimensional scanning method. Theproblem of conformally mapping the two dimensional color informationonto the three dimensional surface model is difficult and it is commonfor mismatching of the color with three-dimensional points to occur.Essentially, where two-dimensional color detectors are used forobtaining the color information, it is difficult to accurately associatecolor information from the detectors with the correct points on thethree dimensional surface model, particularly where relative movementbetween the object and the device occurs between the acquisition of thethree-dimensional topological data and acquisition of thetwo-dimensional image data.

EP 837 659 describes a process and device for obtaining a threedimensional image of teeth. Three-dimensional surface data is obtainedby first covering the surface with an opaque, diffusely reflectingmaterial, and the object is illuminated with monochromatic light. Theimage of the object under the layer is obtained by the process describedin U.S. Pat. No. 4,575,805 using intensity pattern techniques. In orderto obtain a two-dimensional color image of the object, the reflectinglayer has to be removed. The method thus requires the camera to bemanually re-aligned so that the two-dimensional color image should moreor less correspond to the same part of the object as the threedimensional image. Then, the three dimensional image may be viewed on ascreen as a two-dimensional image, and it is possible to superimpose onthis two-dimensional image the two-dimensional color image of the teethtaken by the camera.

U.S. Pat. No. 6,594,539 provides an intraoral imaging system thatproduces images of a dental surface, including three dimensional surfaceimages and also two dimensional color images, with the same camera.

In U.S. Pat. No. 5,440,393, the shape and dimensions of a dentalpatients mouth cavity including upper and lower tooth areas and the jawstructure, are measured by an optical scanner using an externalradiation source, whose reflected signals are received externally andconverted into electronic signals for analysis by a computer. Bothsurface radiation and reflection from translucent internal surfaces arescanned, and processing of reflections may involve a triangulationsystem or holograms.

In U.S. Pat. No. 5,864,640, a scanner is described having a multipleview detector responsive to a broad spectrum of visible light. Thedetector is operative to develop several images of a three dimensionalobject to be scanned. The images are taken from several relative angleswith respect to the object. The images depict several surface portionsof the object to be scanned. A digital processor, coupled to thedetector, is responsive to the images and is operative to develop with acomputational unit 3-D coordinate positions and related imageinformation of the surface portions of the object, and provides 3-Dsurface information that is linked to color information without need toconformally map 2-D color data onto 3-D surface.

Of general background interest, U.S. Pat. Nos. 4,836,674, 5,690,486,6,525,819, EP 0367647 and U.S. Pat. No. 5,766,006 describe devices formeasuring the color of teeth.

SUMMARY OF THE INVENTION

In accordance with the present invention, a device and method fordetermining the surface topology and color of at least a portion of athree dimensional structure is provided. Preferred non-limitingembodiments of the invention are concerned with the imaging of athree-dimensional topology of a teeth segment, optionally including suchwhere one or more teeth are missing. This may allow the generation ofdata for subsequent use in design and manufacture of, for example,prosthesis of one or more teeth for incorporation into said teethsegment. Particular examples are the manufacture of crowns, bridgesdental restorations or dental filings. The color and surface data isprovided in a form that is highly manipulable and useful in manyapplications including prosthesis color matching and orthodontics, amongothers.

The determination of the 3D surface topology of a portion of athree-dimensional structure is preferably carried out using a confocalfocusing method, comprising:

-   -   (a) providing an array of incident light beams propagating in an        optical path leading through a focusing optics and a probing        face; the focusing optics defining one or more focal planes        forward said probing face in a position changeable by said        optics, each light beam having its focus on one of said one or        more focal plane; the beams generating a plurality of        illuminated spots on the structure;    -   (b) detecting intensity of returned light beams propagating from        each of these spots along an optical path opposite to that of        the incident light;    -   (c) repeating steps (a) and (b) a plurality of times, each time        changing position of the focal plane relative to the structure;        and    -   (d) for each of the illuminated spots, determining a        spot-specific position, being the position of the respective        focal plane, yielding a maximum measured intensity of a        respective returned light beam; and based on the determined        spot-specific positions, generating data representative of the        topology of said portion.

The determination of the spot-specific positions in fact amounts todetermination of the in-focus distance. The determination of thespot-specific position may be by measuring the intensity per se, ortypically is performed by measuring the displacement (S) derivative ofthe intensity (I) curve (dI/dS) and determining the relative position inwhich this derivative function indicates a maximum intensity. The term“spot-specific position (SSP)” will be used to denote the relativein-focus position regardless of the manner in which it is determined. Itshould be understood that the SSP is always a relative position as theabsolute position depends on the position of the sensing face. Howeverthe generation of the surface topology does not require knowledge of theabsolute position, as all dimensions in the cubic field of view areabsolute.

The SSP for each illuminated spot will be different for different spots.The position of each spot in an X-Y frame of reference is known and byknowing the relative positions of the focal plane needed in order toobtain maximum intensity (namely by determining the SSP), the Z or depthcoordinate can be associated with each spot and thus by knowing theX-Y-Z coordinates of each spot the surface topology can be generated.

In order to determine the Z coordinate (namely the SSP) of eachilluminated spot the position of the focal plane may be scanned over theentire range of depth or Z component possible for the measured surfaceportion. Alternatively the beams may have components, each of which hasa different focal plane. Thus, by independent determination of SSP forthe different light components, e.g. 2 or 3 with respectivecorresponding 2 or 3 focal planes, the position of the focal planes maybe changed by the focusing optics to scan only part of the possibledepth range, with all focal planes together covering the expected depthrange. Alternatively, the determination of the SSP may involve a focalplane scan of only part of the potential depth range and for illuminatedspots where a maximum illuminated intensity was not reached, the SSP isdetermined by extrapolation from the measured values or othermathematical signal processing methods. Thus, in each case, a Z-value isobtained for each point along an X-Y grid representing a plurality oflight beams. In this manner, a three-dimensional (3D) numerical entity Emay be crated, comprising a plurality of coordinates (X, Y, Z)representative of the surface topology of the object being scanned.

Alternatively, any other suitable method may be employed to obtain the3D entity E.

According to the present invention, a two dimensional (2D) color imageof the 3D structure that is being scanned is also obtained, buttypically within a short time interval with respect to the 3D scan.Further, the 2D color image is taken at substantially the same angle andorientation with respect to the structure as was the case when the 3Dscan was taken. Accordingly, there is very little or no substantialdistortion between the X-Y plane of 3D scan, and the plane of the image,i.e., both planes are substantially parallel, and moreover substantiallythe same portion of the structure should be comprised in both the 3Dscan and the 2D image. This means that each X-Y point on the 2D imagesubstantially corresponds to a similar point on the 3D scan having thesame relative X-Y values. Accordingly, the same point of the structurebeing scanned has substantially the same X-Y coordinates in both the 2Dimage and the 3D scan, and thus the color value at each X, Y coordinateof the 2D color scan may be mapped directly to the spatial coordinatesin the 3D scan having the same X, Y coordinates, wherein to create anumerical entity I representing the color and surface topology of thestructure being scanned.

Where the X, Y coordinates of the color image do not preciselycorrespond to those of the 3D scan, for example as may arise where oneCCD is for the 3D scanning, while another CCD is used for the 2D colorimage, suitable interpolation methods may be employed to map the colordata to the 3D spartial data.

To provide a more accurate mapping, it is possible to construct a 2Dimage along the X-Y plane of the 3D model, and using procedures such asoptical recognition, manipulate the color 2D image to best fit over this3D image. This procedure may be used to correct for any slightmisalignment between the 2D color scan and the 3D scan. Once the color2D image has been suitably manipulated, the color values of the color 2Dimage are mapped onto the adjusted X-Y coordinates of the 3D scan.

Thus the present invention provides a relatively simple and effectiveway for mapping 2D color information onto a 3D surface model.

The present invention thus provides a device and method for obtaining anumerical entity that represents the color and surface topology of anobject. When applied particularly to the intraoral cavity, the device ofthe invention provides advantages over monochrome 3D scaners, includingsuch scanners that are based on confocal focusing techniques. Forexample, the 2D color image capability on its own enables the dentalpractitioner to identify the area of interest within the oral cavitywith a great degree of confidence in order to better aim the device forthe 3D scanning. In other words, an improved viewfinder is automaticallyprovided. Further, rendition of a full color 3D image of the target areacan help the practitioner to decide on the spot whether the scan issufficiently good, or whether there are still parts of the teeth or softtissues that should have been included, and thus help the practitionerto deciode whether or not to acquire another 3D color entity.

Creation of a color 3D entity that is manipulable by a computer isextremely useful in enabling the practictioner to obtain data from suchan entity that is useful for procedures carried out in the dentalcavity.

Thus, according to the present invention, a device is provided fordetermining the surface topology and associated color of at least aportion of a three dimensional structure, comprising:

-   -   scanning means adapted for providing depth data of said portion        corresponding to a two-dimensional reference array substantially        orthogonal to a depth direction;    -   imaging means adapted for providing two-dimensional color image        data of said portion associated with said reference array;    -   wherein the device is adapted for maintaining a spatial        disposition with respect to said portion that is substantially        fixed during operation of said scanning means and said imaging        means. In other words, operation of the scanning means and the        imaging means is substantially or effectively simultaneous in        practical terms, and thus the actual time interval that may        exist between operation of the two means is so short that the        amplitude of any mechanical vibration of the device or movement        of the oral cavity will be so small as can be ignored.

The device is adapted for providing a time interval between acquisitionof said depth data and acquisition of said color image data such thatsubstantially no significant relative movement between said device andsaid portion occurs. The time interval may be between about 0 seconds toabout 100 milliseconds, for example 5, 10, 20, 30, 40, 50, 60, 70, 80,90 or 100 milliseconds, and preferably between about 0 to about 50milliseconds, and more preferably between about 0 and 20 milliseconds.

The device further comprise processing means for associating said colordata with said depth data for corresponding data points of saidreference array. In described embodiments, the operation of saidscanning means is based on confocal imaging techniques. Such scanningmeans may comprise:

-   -   a probing member with a sensing face;    -   first illumination means for providing a first array of incident        light beams transmitted towards the structure along an optical        path through said probing unit to generate illuminated spots on        said portion along said depth direction, wherein said first        array is defined within said reference array;    -   a light focusing optics defining one or more focal planes        forward said probing face at a position changeable by said        optics, each light beam having its focus on one of said one or        more focal plane;    -   a translation mechanism for displacing said focal plane relative        to the structure along an axis defined by the propagation of the        incident light beams;    -   a first detector having an array of sensing elements for        measuring intensity of each of a plurality of light beams        returning from said spots propagating through an optical path        opposite to that of the incident light beams;    -   a processor coupled to said detector for determining for each        light beam a spot-specific position, being the position of the        respective focal plane of said one or more focal planes yielding        maximum measured intensity of the returned light beam, and based        on the determined spot-specific positions, generating data        representative of the topology of said portion.

The first array is arranged to provide depth data at a plurality ofpredetermined spatial coordinates substantially corresponding to thespatial disposition of said incident light beams.

The first illumination means comprises a source emitting a parent lightbeam and a beam splitter for splitting the parent beam into said arrayof incident light beams. The first illumination means may comprise agrating or microlens array.

The device may comprise a polarizer for polarizing said incident lightbeams are polarized. Further, the device may comprise a polarizationfilter for filtering out from the returned light beams light componentshaving the polarization of the incident light beams.

The illumination unit may comprise at least two light sources and eachof said incident beams is composed of light components from the at leasttwo light sources. The at least two light sources emit each a lightcomponent of different wavelength. The light directing optics defines adifferent focal plane for each light component and the detectorindependently detects intensity of each light components.

The at least two light sources may be located so as to define opticalpaths of different lengths for the incident light beams emitted by eachof the at least two light sources.

Typically, the focusing optics operates in a telecentric confocal mode.Optionally, the light directing optics comprises optical fibers.

Typically, the sensing elements are an array of charge coupled devices(CCD). The detector unit may comprise a pinhole array, each pinholecorresponding to one of the CCDs in the CCD array.

The operation of said imaging means may be based on:

-   -   illuminating said portion with three differently-colored        illumination radiations, the said illuminations being combinable        to provide white light,    -   capturing a monochromatic image of said portion corresponding to        each said illuminating radiation, and    -   combining the monochromatic images to create a full color image,    -   wherein each said illuminating radiation is provided in the form        of a second array of incident light beams transmitted towards        the portion along an optical path through said probing unit to        generate illuminated spots on said portion along said depth        direction, wherein said second array is defined within said        reference frame.

The second array is arranged to provide color data at a plurality ofspatial coordinates substantially corresponding to the spatialcoordinates of said first array. The device may comprise colorillumination means adapted for providing three second illuminatingradiations, each of a different color. The color illumination meanscomprises second illumination means for providing said three secondilluminating radiations, each of a different color. Alternatively, thecolor illumination means comprises second illumination means forproviding two said second illuminating radiations, and wherein saidfirst illumination means provides another said second illuminatingradiation each said second illuminating radiation being of a differentcolor. Optionally, each one of said second illumination radiations is adifferent one of red, green or blue light. The second illumination meansmay comprise radiation transmission elements that are configured to belocated out of the path of said light beams or said returned light beamat least within said light focusing optics. The probing member may bemade from a light transmissive material having an upstream opticalinterface with said light focusing optics and a reflective face forreflecting light between said optical interface and said sensing face.The second illumination means may be optically coupled to said opticalinterface for selectively transmitting illuminating radiations in atleast two colors to said portion via said sensing face. The colorillumination means may comprise second illumination means for providingtwo said second illuminating radiations, and wherein said firstillumination means provides another said second illuminating radiationeach said second illuminating radiation being of a different color. Theprobing member may comprise a removable sheath having an inner surfacesubstantially complementary to an outer surface of said probing member,and having a window in registry with said sensing face, wherein saidsheath is made from a waveguiding material and is adapted to transmitsaid light from said second illuminating means from an upstream facethereof to a downstream face associated with said window. The secondillumination means may be optically coupled to said upstream face forselectively transmitting said second illuminating radiations in at leasttwo colors to said portion via said downstream face. Preferably, thesheath is disposable after use with a patient.

In another embodiment, the reflective face comprises a dichroic coating,having relatively high reflectivity and low optical transmissionproperties for a said second illuminating radiation provided by saidfirst illumination means, and relatively low reflectivity and highoptical transmission properties for the two said second illuminatingradiations provided by said second illumination means.

The second illumination means may be adapted for providing secondilluminating radiations within said light focusing optics. Inparticular, the second illumination means may be adapted for providingsecond illuminating radiations at an aperture stop plane of said lightfocusing optics. The second illumination means may be provided on abracket having an aperture configured to allow said light beams and saidreturning light beams to pass therethrough without being opticallyaffected by said bracket.

Optionally, the device further comprises:

-   -   a first polarizing element located just downstream of said        illumination means so as to polarize the light emitted        therefrom;    -   a second polarizing element located just upstream of said first        detector, wherein said second polarizing element is crossed with        respect to the first polarizing element; and    -   a quarter waveplate at the downstream end of said device.

Further optionally the second illumination means are adapted forselective movement in the depth direction.

The device may comprise a mirror inclined to the optical axis of saidlight focusing optics and having, an aperture configured to allow saidlight beams and said returning light beams to pass therethrough withoutbeing optically affected by said mirror, and wherein said secondillumination means comprises at least one white illumination sourceoptically coupled with suitable color filters, said filters selectivelyproviding illumination radiation in each color in cooperation with saidwhite illumination source, wherein said mirror is coupled to said whiteillumination source to direct radiation therefrom along said opticalaxis. The white illumination source may comprise a phosphorus InGaN LED.The filters may be arranged on sectors of a rotatable disc coupled to amotor, predetermined selective angular motion of said disc selectivelycouples said white illumination source to each said filter in turn.

Optionally, the second illumination means are in the form of suitableLED's, comprising at least one LED for providing illumination radiationin each color. Optionally, the second illumination means are in the formof suitable LED's, comprising at least one white illumination sourceoptically coupled with suitable color filters, said filters selectivelyproviding illumination radiation in each color in cooperation with saidwhite illumination source. The white illumination source may comprise aphosphorus InGaN LED. The filters may be arranged on sectors of arotatable disc coupled to a motor, predetermined selective angularmotion of said disc selectively couples said white illumination sourceto each said filter in turn. The device may further comprise a pluralityof optical fibers in optical communication with said filters and withradiation transmission elements comprised in said second illuminationmeans.

The first detector is adapted for selectively measuring intensity ofeach said second illuminating radiation after reflection from saidportion.

Alternatively, the operation of said imaging means is based onilluminating said portion with substantially white illuminationradiation, and capturing a color image of said portion, wherein saidwhite illuminating radiation is provided in the form of a second arrayof incident light beams transmitted towards the portion along an opticalpath through said probing unit to generate illuminated spots on saidportion along said depth direction, wherein said second array is definedwithin said reference frame. The second array is arranged to providecolor data at a plurality of spatial coordinates substantiallycorresponding to the spatial coordinates of said first array. Theimaging means comprises:—

-   -   white illumination radiation means;    -   second detector having an array of sensing elements for        measuring intensity of said white illuminating radiation after        reflection from said portion.

Alternatively, the operation of said imaging means is based onilluminating said portion with substantially white illuminationradiation, selectively passing radiation reflected from said portionthrough a number of color filters, capturing a monochromatic image ofsaid portion corresponding to each said filter, and combining themonochromatic images to create a full color image, wherein saidilluminating radiation is provided in the form of a second array ofincident light beams transmitted towards the portion along an opticalpath through said probing unit to generate illuminated spots on saidportion along said depth direction, wherein said second array is definedwithin said reference frame. The second array is arranged to providecolor data at a plurality of spatial coordinates substantiallycorresponding to the spatial coordinates of said first array.

Alternatively, the operation of said imaging means is based onilluminating said portion with three differently-colored illuminationradiations, capturing a monochromatic image of said portioncorresponding to each said illuminating radiation, and combining themonochromatic images to create a full color image, wherein each saidilluminating radiation is provided in the form of a second array ofincident light beams transmitted towards the portion along an opticalpath through said probing unit to generate illuminated spots on saidportion along said depth direction, wherein said second array is definedwithin said reference frame, and wherein said illuminating radiationsare provided by said first illumination source. The second array isarranged to provide color data at a plurality of spatial coordinatessubstantially corresponding to the spatial coordinates of said firstarray.

The device may further comprise a tri-color sequence generator forcontrolling the illumination of said portion with said secondilluminating radiations.

The device further comprises a processor coupled to said detector forconformally mapping color data provided by said imaging means to saiddepth data provided by said scanning means for each said spatialcoordinates of said first array to provide a color three-dimensionalnumerical entity comprising a plurality of data points, each data pointcomprising three-dimensional surface coordinate data and color dataassociated therewith. The device may also optionally comprise a unit forgenerating manufacturing data for transmission to CAD/CAM device basedon said entity, and a communication port of a communication medium.

The device is adapted for determining color and surface topology of ateeth portion, but may be used for determining color and surfacetopology of any suitable surface.

The present invention is also directed to a method for determining thesurface topology and associated color of at least a portion of a threedimensional structure, comprising:

-   -   (a) providing depth data of said portion corresponding to a        two-dimensional reference array substantially orthogonal to a        depth direction;    -   (b) providing two-dimensional image data of said portion        associated with said reference array;    -   (c) ensuring that a spatial disposition with respect to said        portion during steps (a) and (b) is substantially fixed;    -   (d) conformally mapping said color data to said depth data for        said reference array.

Preferably, in step (c), a minimum time interval is allowed betweenacquisition of said depth data and acquisition of said image data. Thetime interval may be between about 0 seconds to about 100 milliseconds,preferably between 0 and 50 milliseconds, and more preferably between 0and 20 milliseconds.

In described embodiments, the depth data is provided using confocalimaging techniques. The method can then comprise:

-   -   (i) providing a first array of incident light beams defined        within said reference array propagating in an optical path        leading through a focusing optics and through a probing face;        the focusing optics defining one or more focal planes forward        said probing face in a position changeable by said optics, each        light beam having its focus on one of said one or more focal        plane; the beams generating a plurality of illuminated spots on        the structure;    -   (ii) detecting intensity of returned light beams propagating        from each of these spots along an optical path opposite to that        of the incident light;    -   (iii) repeating steps (i) and (ii) a plurality of times, each        time changing position of the focal plane relative to the        structure;    -   (iv) for each of the illuminated spots, determining a        spot-specific position, being the position of the respective        focal plane yielding a maximum measured intensity of a        respective returned light beam; and    -   (v) generating data representative of the topology of said        portion.

Step (ii) may be based on illuminating said portion with at least threedifferently-colored illumination radiations, said illuminationradiations being combinable to produce white radiation, capturing amonochromatic image of said portion corresponding to each saidilluminating radiation, and combining the monochromatic images to createa full color image, wherein each said illuminating radiation is providedin the form of a second array of incident light beams transmittedtowards the portion along an optical path through said probing unit togenerate illuminated spots on said portion along said depth direction,wherein said second array is defined within said reference frame. Thesecond array is arranged to provide color data at a plurality of spatialcoordinates substantially corresponding to the spatial coordinates ofsaid first array.

Optionally, the sources for the at least three colored illuminations maybe located at the confocal system aperture stop, and facing theobjective lens of the system. Preferably, the illumination sources areconfigured to have a relatively low numerical aperture compared withthat of the first array of light beams. Further preferably, the confocalsystem is configured for chromatically dispersing said coloredilluminations therethrough.

Preferably, the method further comprises providing an improved focus 2Dcolor image of said structure, comprising:—

-   -   (I) sequentially illuminating the structure with each one of a        plurality of illuminations, each said illumination having a        different wavelength in the visible spectrum;    -   (II) providing a monochrome image of the structure when        illuminated with each illumination in (I);    -   (III) manipulating image data obtained in (II) to provide a best        focus composite image;    -   (IV) manipulating image data in (II) and (III) to provide a        composite focused color image of the structure.

Further preferably, the said sources for the colored illuminations aremoveable in the depth direction.

Optionally, the method of the invention further comprises the steps of:

-   -   polarizing the emitted colored illuminations by means of a first        polarizing element;    -   modifying the said polarized color illuminations on the way to        the structure and on their return therefrom by means of a        quarter waveplate;    -   causing the returning color illuminations to pass through a        second polarizing element located just upstream of said first        detector, wherein said second polarizing element is crossed with        respect to the first polarizing element.

Step (ii) may be based on illuminating said portion with substantiallywhite illumination radiation, selectively passing radiation reflectedfrom said portion through a number of color filters, capturing amonochromatic image of said portion corresponding to each said filter,and combining the monochromatic images to create a full color image,wherein said illuminating radiation is provided in the form of a secondarray of incident light beams transmitted towards the portion along anoptical path through said probing unit to generate illuminated spots onsaid portion along said depth direction, wherein said second array isdefined within said reference frame. The second array is arranged toprovide color data at a plurality of spatial coordinates substantiallycorresponding to the spatial coordinates of said first array.

Step (ii) may be based on illuminating said portion with threedifferently-colored illumination radiations, capturing a monochromaticimage of said portion corresponding to each said illuminating radiation,and combining the monochromatic images to create a full color image,wherein each said illuminating radiation is provided in the form of asecond array of incident light beams transmitted towards the portionalong an optical path through said probing unit to generate illuminatedspots on said portion along said depth direction, wherein said secondarray is defined within said reference frame, and wherein saidilluminating radiations are provided by said first illumination source.The second array is arranged to provide color data at a plurality ofspatial coordinates substantially corresponding to the spatialcoordinates of said first array.

The data representative of said topology may be used for constructing anobject to be fitted within said structure, or may be converted into aform transmissible through a communication medium to recipient.Typically, the structure is a teeth segment. The structure may be ateeth segment with at least one missing tooth or a portion of a toothand said object is said at least one missing tooth or the portion of thetooth. Thus, for example, steps (i) to (v) may be repeated for twodifferent surfaces of said structure to provide surface topologiesthereof, and the surface topologies may then be combined to obtain colorand topological data representative of said structure.

The method of the invention, and also the operation of the device of thepresent invention, may be modified to take account of any possiblerelative movement between the device and the intra oral cavity, forexample as follows:—

-   -   (a) providing depth data of said portion corresponding to a        two-dimensional reference array substantially orthogonal to a        depth direction;    -   (b) providing two-dimensional image data of said portion        associated with said reference array;    -   (c) repeating step (a);    -   (d) for each image color data point obtained in step (b), i.e.,        for each particular (x, y) point on the array for which a color        value was obtained in step (b), providing an estimated value for        depth, based on the depth values obtained in steps (a) and (c)        for the same part of the array, i.e. based on the Z-values        obtained for the same (x, y) point in steps (a) and (c). The        estimated value may be based on a simple arithmetic mean, on a        weighted mean, or on any suitable empirical or theoretical        formula, algorithm and so on.

Of course, step (a) may be repeated a number of times consecutivelybefore step (b), and optionally also after step (b), the time intervalsbetween each step being taken. In any case, for each point on the array(x, y), the values of depth Z may be plotted against elapsed time, forsteps (a) (single or repeated), through step (b) and steps (c) (singleor repeated), and the best estimate of the value of Z corresponding tothe time interval when step (b) was carried out can be calculated,using, for example, any suitable interpolation or curve-fitting method.

Alternatively, the method of the invention, and thus the operation ofthe device of the present invention, may be modified to take account ofany possible relative movement between the device and the intra oralcavity, for example as follows:

-   -   (a) providing two-dimensional image data of said portion        associated with said reference array;    -   (b) providing depth data of said portion corresponding to a        two-dimensional reference array substantially orthogonal to a        depth direction;    -   (c) repeating step (a);    -   (d) for each depth data point obtained in step (b), i.e., for        each particular (x, y) point on the array for which a depth        value was obtained in step (b), providing an estimated value for        color, based on the color values obtained in steps (a) and (c)        for the same part of the array, i.e. based on the C-values        obtained for the same (x, y) point in steps (a) and (c). The        estimated value may be based on a simple arithmetic mean, on a        weighted mean, or on any suitable empirical or theoretical        formula, algorithm and so on.

Of course, step (a) may optionally be repeated a number of timesconsecutively before step (b), and optionally also after step (b), thetime intervals between each step being taken. In any case, for eachpoint on the array (x, y), the values of color C may be plotted againstelapsed time, for steps (a) (single or repeated), through step (b) andsteps (c) (single or repeated), and the best estimate of the value of Ccorresponding to the time interval when step (b) was carried out can becalculated, using, for example, any suitable interpolation orcurve-fitting method.

Optionally, the steps of providing color values and depth values may berepeated in any sequence, for example in alternate sequence, and asuitable color value may be associated with a corresponding depth value,similarly to the manner described above, mutatis mutandis.

The invention also relates to a method for reconstruction of color andtopology of a three-dimensional structure comprising:

-   -   determining surface topologies from at least two different        positions or angular locations relative to the structure, by the        method of the invention described above;    -   combining the surface topologies to obtain color and topological        data representative of said structure.

The method may be applied to the reconstruction of topology of a teethportion, and comprise the steps:

-   -   determining surface topologies of at least a buccal surface and        a lingual surface of the teeth portion;    -   combining the surface topologies to obtain data representative        of a three-dimensional structure of said teeth portion.

The method may be applied to obtaining data representative of athree-dimensional structure of a teeth portion with at least one missingtooth or a portion of a tooth.

The data may be used in a process of designing or manufacturing of aprostheses of said at least one missing tooth or a portion of a tooth.Such a prosthesis may be, for example, a crown, a bridge, a dentalrestoration or a dental filing.

The present invention is directed to a method of providing data usefulin procedures associated with the oral cavity comprising:

-   -   providing at least one numerical entity representative of the        three-dimensional surface geometry and colour of at least part        of the intra-oral cavity; and    -   manipulating said entity to provide desired data therefrom.

Typically, the numerical entity comprises surface geometry and colourdata associated with said part of the intra-oral cavity, and the colourdata includes actual or perceived visual characteristics including hue,chroma, value, translucency, reflectance.

In an first embodiment, particularly useful for differentiating a firsttissue from a second tissue, wherein said first tissue comprisessubstantially different colour characteristics from those of said secondtissue, comprises

-   -   separating said surface geometry and colour data into at least        two tissue data sets, wherein    -   a first said tissue data set comprises surface geometry and        colour data, wherein said colour data thereof is correlated with        a colour representative of said first tissue; and    -   a second said tissue data set comprises surface geometry and        colour data, wherein said colour data thereof is correlated with        a colour representative of said second tissue.

The first tissue comprises hard tissues such as teeth, and the softtissue comprises at least one of gums, tongue, cheeks and lips.

The first tissue data set may correspond to a plurality of teeth of saidintraoral cavity, and in the next step the first tissue data set isdivided into a plurality of sub data sets, wherein each said sub dataset correspond to a different said tooth.

In the next step, the sub data sets may be manipulated in a mannersimulating an orthodontic treatment on said teeth.

Optionally, the sub data sets may be displayed as images correspondingto individual teeth.

The first embodiment may also be used for determining the finish linefor a dental preparation.

The second embodiment is particularly useful for stitching at least twosaid entities, wherein at least a portion of said entities compriseoverlapping spatial data, comprising:—

-   -   for each entity providing at least one sub entity comprising a        first tissue data set comprising surface geometry and colour        data, wherein said colour data thereof is correlated with a        colour representative of a first tissue; and    -   stitching said first tissue data sets together based on        registering    -   portions of said data set comprising said overlapping spatial        data.

The first tissue may comprise hard tissues of the intraoral cavity, suchas

-   -   for example teeth.

Optionally, the method may further comprise the step of:

-   -   for each entity separating said surface geometry and colour data        into a second tissue data set, comprising surface geometry and        colour data, wherein said colour data thereof is correlated with        a colour representative of a second tissue.

The second tissue typically comprises the soft tissues of the intraoralcavity, including at least one of gums, tongue, cheeks and lips.

In one variation of the method, step (b) comprises:

-   -   providing coarse stitching of the original entities of step (a)        by registering overlapping spatial data thereof to determine        coarse spatial relationships between said entities;    -   applying said coarse spatial relationships to said first tissue        data sets to facilitate registration of overlapping portions;        and    -   stitching said first tissue data sets together based on        registering said overlapping portions.

By eliminating the data associated with the soft tissues, and proceedingwith stitching only the hard tissues using the colour data, the qualityof the stitching procedure is significantly better than when using thefull infra-oral data for the stitching procedure.

A third embodiment is particularly useful for providing a finish linefor a preparation area in said intraoral cavity, though it may also beused for virtually separating the teeth from the gums. The methodcomprises:—

-   -   comparing the colour data for each pair of spatially adjacent        data points in said numerical entity;    -   if the colour data for said pair of data points are        substantially different    -   one from the other, providing one said data point of said pair        of data points; and    -   (c) applying a search algorithm for identifying said finish line        in said numerical entity, wherein said application of said        algorithm is initially applied to a part of said entity        corresponding to the said provided data points of step (b).

Optionally, in step (b) the data point of said pair of data pointshaving colour data associated with a hard tissue is provided for step(c).

In a fourth embodiment of the invention, the colour data associated withat least one tooth of said intraoral cavity is used for providingshading data for a prosthesis for use in said intraoral cavity.

The method typically includes the steps:

-   -   providing separate numerical sub-entities each associated with a        different one of at least one tooth within said infra-oral        cavity;    -   providing a prosthesis entity comprising surface geometrical        data, said prosthesis entity being associated with a desired        prosthesis;    -   mapping colour data from at least one sub entity in step (a) to        said    -   prosthesis entity according to predetermined criteria.

Optionally, step (c) comprises

-   -   transforming the geometrical data of each said separate        numerical sub-entities to correspond to a predetermined        geometrical form, and mapping said colour data to said        geometrical form to provide for each said separate numerical        sub-entity a transformed sub-entity;    -   transforming the geometrical data in (b) to correspond to said        prosthesis entity and mapping colour data associated with the        transformed sub-entity to said prosthesis entity.

Optionally, in step (a) a single numerical sub-entity associated withone tooth within said infra-oral cavity is provided.

Alternatively, in step (a) a plurality of separate numericalsub-entities associated with a corresponding plurality of teeth withinsaid infra-oral cavity are provided; and wherein in step (c) the saidtransformed sub-entities are combined to a single transformedsub-entity, wherein colour data corresponding to said plurality ofnumerical sub-entities in (a) are combined in a predetermined manner.Such a predetermined manner comprises averaging the colour value at eachcorresponding data point of said plurality of numerical sub-entities in(a). Optionally, the predetermined manner comprises weight averaging thecolour value at each corresponding data point of said plurality ofnumerical sub-entities in (a).

Typically, step (a) comprises:—

-   -   separating said surface geometry and colour data into at least        two tissue data sets, wherein    -   a first said tissue data set comprises surface geometry and        colour data, wherein said colour data thereof is correlated with        a colour representative of said first tissue; and    -   a second said tissue data set comprises surface geometry and        colour data wherein said colour data thereof is correlated with        a colour representative of said second tissue.

The first tissue comprises hard tissues such as teeth, and the softtissue comprises at least one of gums, tongue, cheeks and lips.

The first tissue data set may correspond to a plurality of teeth of saidintraoral cavity, and in the next step the first tissue data set isdivided into a plurality of sub data sets, wherein each said sub dataset correspond to a different said tooth.

In another aspect of the present invention, a computer readable mediumis provided that embodies in a tangible manner a program executable forproviding data useful in procedures associated with the oral cavity. Thecomputer readable medium comprises:

-   -   a first set of data representative of the three dimensional        surface geometry and colour of at least part of the intra oral        cavity;    -   means for manipulating said first data set to provide desired        data therefrom.

The medium may comprise, for example, optical discs, magnetic discs,magnetic tapes, and so on.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present invention will be obtained by reference to thefollowing detailed description that sets forth illustrative embodiments,in which the principles of the invention are utilized, and theaccompanying drawings of which:

In order to understand the invention and to see how it may be carriedout in practice, a number of embodiments will now be described, by wayof non-limiting example only, with reference to the accompanyingdrawings, in which:

FIG. 1 illustrates the main elements of embodiments of the devices forcreating a three-dimensional color numerical entity that is manipulatedaccording to the present invention.

FIGS. 2A, 2B, 2C graphically illustrates the creation of a threedimensional color entity from a three dimensional monochrome entity anda two dimensional color entity.

FIG. 3 graphically illustrates an alignment procedure according to theinvention for aligning the X-Y coordinates of a three dimensionalmonochrome entity with corresponding coordinates of a two dimensionalcolor entity.

FIGS. 4A and 4B schematically illustrate the main elements of a portionof a device used for providing a three dimensional monochrome entity.

FIGS. 5A, 5B, 5C illustrate in plan view, side view and isometric view,respectively, a probe used in first embodiment of the device of FIG. 1to provide a two dimensional color entity.

FIG. 6 illustrates in side view a sheath for a probe used in secondembodiment of the device of FIG. 1 to provide a two dimensional colorentity.

FIG. 7A illustrates in side view a probe used in third embodiment of theinvention to provide a two dimensional color entity. FIG. 7B illustratesthe transmission and reflection characteristics of a typical dichroiccoating used in the probe of FIG. 7A.

FIG. 8 illustrates in side view the general arrangement of the mainelements used in fourth embodiment of the device of FIG. 1 to provide atwo dimensional color entity.

FIG. 9 illustrates an LED arrangement used with the embodiment of FIG.8.

FIG. 10 illustrates an alternative illumination arrangement used withthe embodiment of FIG. 8. FIG. 10A illustrates details of the tri-colordisc used with the illumination arrangement of FIG. 10.

FIG. 11 illustrates in side view the general arrangement of the mainelements used in fifth embodiment of the device of FIG. 1 to provide atwo dimensional color entity.

FIG. 12 illustrates in side view the general arrangement of the mainelements used in sixth embodiment of the device of FIG. 1 to provide atwo dimensional color entity.

FIG. 13 illustrates in side view the general arrangement of the mainelements used in seventh embodiment of the device of FIG. 1 to provide atwo dimensional color entity.

FIG. 14 illustrates the main steps of the method according to the firstembodiment of the present invention.

FIG. 15 illustrates the main steps of the method according to the secondembodiment of the present invention.

FIG. 16 illustrates the main steps of the method according to the thirdembodiment of the present invention.

FIG. 17 illustrates the main steps of the method according to the fourthembodiment of the present invention.

FIG. 18 illustrates a portion of the intraoral cavity on which it isdesired to implant a dental prosthesis.

FIG. 19 schematically illustrates a transformation step according to themethod of FIG. 17.

FIG. 20 is a block diagram of a system according to an embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE INVENTION

The first step of the method according to the present invention relatesto providing at least one numerical entity that is representative of thethree-dimensional surface geometry and colour of at least part of theintra-oral cavity.

The said numerical entity is typically at least “four-dimensional”, thatis, each data point of the data set comprises at least four primeindependent variables. In the preferred embodiments of the invention,three of the prime independent variables relate to spatial coordinatesof a surface, typically defined along orthogonal Cartesian axes, x, y,z. Alternatively, these variables may be defined along polar axes or anyother geometric system in which a surface may be described. The fourthprime independent variable refers to a colour parameter that isexpressed numerically and associated with the spatial coordinates. Thecolour parameter may itself be comprised of independent prime colourvariables—for example relating to the red, blue and green (RGB)components associated with the colour parameter. Alternatively, thecolour parameter may be expressed in terms of the Hue, Saturation andIntensity (HIS). Alternatively, any other colour parameter may be used,including parameters that provide a measure of internal reflectance andtranslucency, or any other optical property of teeth.

Thus, the numerical entity may comprise a data set of a plurality of4-dimensional arrays (x, y, z, c), wherein each array represents the x,y, z, geometrical coordinates and the colour c of a point on a surfacewithin the intra-oral cavity.

Any suitable means may be used to provide the numerical entity. Forexample, a three-dimensional surface scanner with colour capabilitiesmay be used. Advantageously, such a scanner makes use of confocalimaging for providing an accurate three-dimensional representation ofthe target surface within the intra-oral cavity. Colour values are thenadded to each data point of this data set by obtaining a two-dimensionalcolour image of the target surface, and then mapping the colour valuesof the two-dimensional image onto the three-dimensional “image”.

The following are examples on how to obtain the 3d colour numericalentity.

Reference is first being made to FIG. 1 which illustrates the generalrelationship between the various elements of a device for providing a 3Dnumerical entity, generally designated with the numeral 100, accordingto the embodiments described herein.

The device 100 comprises a main illumination source 31 for illuminatingthe object of interest 26, typically a part of the intraoral cavity, andis optically coupled to main optics 41 to provide depth Z values for anarray range of X-Y points (according to a known frame of reference)along the surface of the object 26. Detection optics 60 comprises animage sensor, typically a CCD, that is preferably monochromatic tomaximize the resolution of the device, and which typically defines theX-Y frame of reference. Alternatively, the CCD may be adapted to receivecolor images. The detection optics 60 receives image data from the mainoptics 41 and the image processor 24 determines the depth Z values foreach X-Y point illuminated on the object 26 based on this image data. Inthis manner, a manipulable three-dimensional numerical entity Ecomprising the surface coordinates of the object 26.

The device 100 further comprises color illuminating means, such as forexample a tri-color sequence generator 74, for selectively illuminatingthe object 26 with suitable colors, typically Green, Red and Blue, andfor each such monochromatic illumination, a two dimensional image of theobject 26 is captured by the detection optics 60. The processor 24 thenprocesses the three differently colored monochromatic images andcombines the same to provide a full color 2D image of the object. Thedevice 100 is configured for providing color data for an array of X-Ypoints that is according to the same frame of reference as the X-Y arrayused for obtaining the 3D entity.

The processor 24 aligns the 2D color image with the 3D entity previouslycreated, and then provides color values to this entity by mapping colorvalues to the entity at aligned X-Y points. Such alignment isstraightforward because both the 3D data and the 2D color data arereferenced to the same X-Y frame of reference. Referring to FIGS. 2A,2B, 2C, the mapping procedure is performed as follows. Athree-dimensional numerical entity E is obtained by-determining depthZ-values for a grid of X-Y points, illuminated via main optics 41 anddetermined by image processor 24. The entity E thus comprises an arrayof (X, Y, Z) points, as illustrated in FIG. 2A. The X-Y plane of entityE is substantially parallel to the sensing face of the image sensingmeans of the detection optics 60, typically a CCD. Almost concurrently,i.e., either just before or just after the readings for determining the3D entity E are obtained by the detection optics 60, a 2D color image ofthe object 26 is taken using the same detection optics 60, atsubstantially the same relative spatial disposition between thedetection optics 60 and the object 26, FIG. 2B. If a monochromatic CCDis used, the 2D color image obtained is a composite created from threeseparate monochromatic images, each provided by illuminating the object26 with a different color, such as for example red, green and blue. The2D color image thus corresponds to another entity N comprised of thelocation and color value of each pixel forming this image, (X′, Y′, C).The X′-Y′ coordinates' of the pixels are on a plane substantiallyparallel to the X-Y plane of the entity E, and furthermore thesecoordinates represent substantially the same part of the object 26 asthe X-Y coordinates of entity E. The reason for this is that the opticalinformation that is used for creating both the 3D entity E and the color2D entity N are obtained almost simultaneously with a very small timeinterval therebetween, and typically there is insufficient time for anysignificant relative movement between the image plane of the detectionoptics 60 and the object 26 to have occurred between the two scans.Thus, similar X-Y and X′-Y′ coordinates in the entities E and N,respectively, will substantially represent the same part of the object26. Accordingly, the color value C of each pixel of entity N can bemapped to the data point of entity E having X-Y coordinates that are thesame as the X′-Y′ coordinates of the pixel, whereby to create anotherentity I comprising surface coordinate and color data, (X, Y, Z, C), asillustrated in FIG. 2C.

Were the relative angle and disposition between the plane of the sensingface of the detection optics 60 with respect to the object 26 changesignificantly between the 2D and the 3D scans, then the X-Y coordinatesof entity E having similar values to the X′-Y′ coordinates of entity Ncould correspond to different parts of the object 26, and thus it maythen be difficult to map the color values of entity N to entity E.However, if only a small movement between the detection optics 60 withrespect to the object 26 occurs, particularly involving a relativetranslation or a rotation about the depth direction (Z), butsubstantially no change in the angular disposition between detectionoptics 60 and the object 26 about the X or Y axes, it may still bepossible to map the color values of entity N to entity E, but first analignment procedure must be followed.

Referring to FIG. 3, such an alignment procedure may be based on opticalcharacter recognition (OCR) techniques. In the X-Y plane, the X-Ycoordinates of entity E can be divided up into two groups, one groupcomprising Z values corresponding to the depth of the object, and asecond group for which no reasonable Z value was found and this groupcorresponds to the background relative to object 26. The profiles ofshapes represented by the X-Y coordinates of the first group of entityE, herein referred to as another entity E′, are then optically comparedwith profiles of shapes corresponding to the X′-Y′ coordinates of entityN, herein referred to as another entity N′. Accordingly, entity E′ istranslated or rotated (coplanarly) with respect to entity N′ until abest fit between the optical shapes between the two entities isobtained, using OCR techniques that are well known in the art.Typically, the image processor, or another computer, will attempt toalign the outer border of the object 26 as seen along the Z-axis andencoded in entity E with optical elements in the 2D color image encodedin entity N. Thereafter, the color value C of each X′-Y′ coordinate ofentity N is mapped to the appropriate data point of entity E having thealigned X-Y coordinates corresponding thereto. The color mappingoperation to create entity I may be executed by any suitablemicroprocessor means, typically processor 24 of the device 100 (FIG.4B).

The main optics 41, main illumination source 31, detection optics 60 andimage processor 24 are now described with reference to FIGS. 4A and 4Bwhich illustrate, by way of a block diagram an embodiment of a system 20for confocal imaging of a three dimensional structure according to WO00/08415 assigned to the present assignee, the contents of which areincorporated herein. Alternatively, any suitable confocal imagingarrangement may be used in the present invention.

The system 20 comprises an optical device 22 coupled to a processor 24.Optical device 22 comprises, in this specific embodiment, asemiconductor laser unit 28 emitting a laser light, as represented byarrow 30. The light passes through a polarizer 32 which gives rise to acertain polarization of the light passing through polarizer 32. Thelight then enters into an optic expander 34 which improves the numericalaperture of the light beam 30. The light beam 30 then passes through amodule 38, which may, for example, be a grating or a micro lens arraywhich splits the parent beam 30 into a plurality of incident light beams36, represented here, for ease of illustration, by a single line. Theoperation principles of module 38 are known per se and the art and theseprinciples will thus not be elaborated herein.

The optical device 22 further comprises a partially transparent mirror40 having a small central aperture. It allows transfer of light from thelaser source through the downstream optics, but reflects lighttravelling in the opposite direction. It should be noted that inprinciple, rather than a partially transparent mirror other opticalcomponents with a similar function may also be used, e.g. a beamsplitter. The aperture in the mirror 40 improves the measurementaccuracy of the apparatus. As a result of this mirror structure thelight beams will yield a light annulus on the illuminated area of theimaged object as long as the area is not in focus; and the annulus willturn into a completely illuminated spot once in focus. This will ensurethat a difference between the measured intensity when out-of- andin-focus will be larger. Another advantage of a mirror of this kind, asopposed to a beam splitter, is that in the case of the mirror internalreflections which occur in a beam splitter are avoided, and hence thesignal-to-noise ratio improves.

The unit further comprises a confocal optics 42, typically operating ina telecentric mode, a relay optics 44, and an endoscopic probing member46. Elements 42, 44 and 46 are generally as known per se. It shouldhowever be noted that telecentric confocal optics avoidsdistance-introduced magnification changes and maintains the samemagnification of the image over a wide range of distances in the Zdirection (the Z direction being the direction of beam propagation). Therelay optics enables to maintain a certain numerical aperture of thebeam's propagation.

The endoscopic probing member 46 typically comprises a rigid,light-transmitting medium, which may be a hollow object defining withinit a light transmission path or an object made of a light transmittingmaterial, e.g. a glass body or tube. At its end, the endoscopic probetypically comprises a mirror of the kind ensuring a total internalreflection and which thus directs the incident light beams towards theteeth segment 26. The endoscope 46 thus emits a plurality of incidentlight beams 48 impinging on to the surface of the teeth section.

Incident light beams 48 form an array of light beams arranged in an X-Yplane, in the Cartesian frame 50, propagating along the Z axis. As thesurface on which the incident light beams hits is an uneven surface, theilluminated spots 52 are displaced from one another along the Z axis, atdifferent (X_(i), Y_(i)) locations. Thus, while a spot at one locationmay be in focus of the optical element 42, spots at other locations maybe out-of-focus. Therefore, the light intensity of the returned lightbeams (see below) of the focused spots will be at its peak, while thelight intensity at other spots will be off peak. Thus, for eachilluminated spot, a plurality of measurements of light intensity aremade at different positions along the Z-axis and for each of such(X_(i), Y_(i)) location, typically the derivative of the intensity overdistance (Z) will be made, the Z_(i) yielding maximum derivative, Z₀,will be the in-focus distance. As pointed out above, where, as a resultof use of the punctured mirror 40, the incident light forms a light diskon the surface when out of focus and a complete light spot only when infocus, the distance derivative will be larger when approaching in-focusposition thus increasing accuracy of the measurement.

The light scattered from each of the light spots includes a beamtravelling initially in the Z-axis along the opposite direction of theoptical path traveled by the incident light beams. Each returned lightbeam 54 corresponds to one of the incident light beams 36. Given theunsymmetrical properties of mirror 40, the returned light beams arereflected in the direction of the detection optics 60. The detectionoptics 60 comprises a polarizer 62 that has a plane of preferredpolarization oriented normal to the plane polarization of polarizer 32.The returned polarized light beam 54 pass through an imaging optic 64,typically a lens or a plurality of lenses; and then through a matrix 66comprising an array of pinholes. CCD camera has a matrix or sensingelements each representing a pixel of the image and each onecorresponding to one pinhole in the array 66.

The CCD camera is connected to the image-capturing module 80 ofprocessor unit 24. Thus, each light intensity measured; in each of thesensing elements of the CCD camera is then grabbed and analyzed, in amanner to be described below, by processor 24.

Unit 22 further comprises a control module 70 connected to a controllingoperation of both semi-conducting laser 28 and a motor 72. Motor 72 islinked to telecentric confocal optics 42 for changing the relativelocation of the focal plane of the optics 42 along the Z-axis. In asingle sequence of operation, control unit 70 induces motor 72 todisplace the optical element 42 to change the focal plane location andthen, after receipt of a feedback that the location has changed, controlmodule 70 will induce laser 28 to generate a light pulse. At the sametime, it will synchronize image-capturing module 80 to grab datarepresentative of the light intensity from each of the sensing elements.Then in subsequent sequences the focal plane will change in the samemanner and the data capturing will continue over a wide focal range ofoptics 44.

Image capturing module 80 is connected to a CPU 82, which thendetermines the relative intensity in each pixel over the entire range offocal planes of optics 42, 44. As explained above, once a certain lightspot is in focus, the measured intensity will be maximal. Thus, bydetermining the Z, corresponding to the maximal light intensity or bydetermining the maximum displacement derivative of the light intensity,for each pixel, the relative position of each light spot along theZ-axis can be determined. Thus, data representative of thethree-dimensional pattern of a surface in the teeth segment, can beobtained. This three-dimensional representation may be displayed on adisplay 84 and manipulated for viewing, e.g. viewing from differentangles, zooming-in or out, by the user control module 86 (typically acomputer keyboard).

The device 100 further comprises means for providing a 2D color image ofthe same object 26, and any suitable technique may be used for providingthe color image. A number of such techniques are described below.

The first technique is based on illuminating the object 26 sequentiallywith three different colored lights such as red, green and blue, andcapturing a monochromatic image corresponding to each color via CCD 68and the image capture device 80 (see FIGS. 4A, 4B). Referring to FIG. 1,tri-color light sources 71, i.e., one or more light sources that provideilluminating radiations to the object 26 in a plurality of differentcolors, are coupled to a tri-color sequence generator 74, which aresuitably controlled by the processing unit 24 to provide the threecolored illuminations via delivery optics 73 in a predeterminedsequence. The colored illuminations are provided at a relative shorttime interval, typically in the range of about 0 to 100 milliseconds, insome cases being in the order of 50 milliseconds or 20 milliseconds, forexample, with respect to the 3D scan, directly before or after the same.Suitable processing software 82 combines the three images to provide a2D color image comprising an array of data points having location (X, Y)and color (C) information for each pixel of the 2D color image.

According to a first embodiment of the device 100, the delivery optics73 is integral with endoscope 46, which is in the form of a probingmember 90, as illustrated in FIGS. 5A, 5B and 5C. The probing member 90is made of a light transmissive material, typically glass and iscomposed of an anterior segment 91 and a posterior segment 92, tightlyglued together in an optically transmissive manner at 93. Slanted face94 is covered by a totally reflective mirror layer 95. Glass disk 96defining a sensing surface 97 may be disposed at the bottom in a mannerleaving an air gap 98. The disk is fixed in position by a holdingstructure which is not shown. Three light rays are 99 from the mainoptics 42 are represented schematically. As can be seen, they bounce atthe walls of the probing member at an angle in which the walls aretotally reflective and finally bounce on mirror 95 and reflected fromthere out through the sensing face 97. The light rays focus on focusingplane 101, the position of which can be changed by the focusing optics(not shown in this figure). The probe member 90 comprises an interface78 via which optical communication is established with the relay optics44 and the remainder of the, device 100. The probe 90 further comprisesa plurality of tri-color LED's 77, for providing the coloredillumination to the object 26.

The LED's 77 typically comprise different LED's for providing blueradiation and green radiation when red illuminating radiation is used asthe illumination source 31 for the main optics 41 when creating the 3Dentity. Alternatively, if a blue illuminating radiation is used as theillumination source 31, the LED's 77 may comprise green and red LED's,and if a green illuminating radiation is used as the illumination source31, LED's 77 may comprise blue and red LED's.

The tri-color LED's 77 are each capable of providing an illuminationradiation in one of three colors, typically red, green or blue, ascontrolled via the tri-color sequence generator. Alternatively, aplurality of LED's in three groups, each group providing illumination inone of the desired colors, may be provided. The LED's 77 are located atthe periphery of the interface 78 such that the LED's do not interferewith the other optical operations of the device 100. In particular suchoperations include the transmission of the illuminating radiation forthe confocal focusing operations, and also the transmission of reflectedlight from the object 26 to the main optics 41 to provide the 3D entityor the 2D color entity. The LED's are mounted substantially orthogonallywith respect to the interface 78, and thus, as illustrated in FIG. 5C,light from each of the LED's 77 is transmitted by internal reflectionwith respect to the walls of the probe 90, to the user interface end 79of the probe.

According to a second embodiment of the device 100, the endoscope 46, isalso in the form of a probing member 90, substantially as described withrespect to the first embodiment, but with the difference that there areno LED's directly mounted thereon at the interface 78, mutatis mutandis.In the second embodiment the delivery optics 73 is in the form of adisposable sleeve, shroud or sheath 190 that covers the outer surfacethe probing member 90, as illustrated in FIG. 6. The sheath 190 is madefrom a waveguiding material, such as an acrylic polymer for example,capable of transmitting an illuminating radiation from the upstream face191 of the sheath 190 therethrough and to the downstream face 192thereto. The upstream face 191 is in the form of a peripheral surfacearound the interface 78. The downstream face 192 is formed as aperipheral projection surrounding a window 193 comprised in said sheath190. The window 193 is in registry with the user interface end 79 of theprobe 90. A plurality of tri-color LED's 177 for providing the coloredillumination to the object 26 are mounted on the device 100 justupstream of the sheath 190. The tri-color LED's 177 are each capable ofproviding an illumination radiation in one of three colors, typicallyred, green or blue, as controlled via the tri-color sequence generator74. Alternatively, a plurality of LED's in three groups, each groupproviding one colored illumination, may be provided. The LED's 177 arelocated outside of the main optics of the device 100, and thus the LED'sdo not interfere with the other optical operations of the device 100 inparticular including the transmission of the illuminating radiation forthe confocal focusing operations, or in the transmission of reflectedlight from the object 26 to provide the 3D entity or the 2D colorentity. The LED's are mounted substantially opposite to the upstreamface 191, and thus, as illustrated in FIG. 6, light from each of theLED's 177 is transmitted by the waveguiding sheath 190 to downstreamface 192 and thence to the object 26. In this embodiment, the sheath 190is particularly useful in maintaining hygienic conditions between onepatient and the next, and avoids the need for sterilizing the probe 90,since the sheath may be discarded after being used with one patient, andreplaced with another sterilised sheath before conducting an intra-oralcavity survey with the next patient.

In either one of the first or second embodiments, or variations thereof,a red laser may be used as the illumination source 28 for the mainoptics when creating the 3D entity. As such, this illumination means mayalso be used to obtain the red monochromatic image for the creation ofthe 2D color image, by illuminating the object 26 and recording theimage with the optical detector 60. Accordingly, rather than tri-colorLED's or LED's or three different colors, it is only necessary toprovide LED's adapted to provide only the remaining two colors, greenand blue. A similar situation arises if the illumination source for themain optics 41 is a green or blue laser, wherein illuminating radiationsin only the remaining two colors need to be provided, mutatis mutandis.

In these embodiments, the positioning of the illumination sources at theupstream end of the probe 90 where there is ample room rather than atthe patient interface end 79 where space is tight.

According to a third embodiment of the device 100, the endoscope 46 isalso in the form of a probing member 90, substantially as described withrespect to the second embodiment with the following differences, mutatismutandis. As illustrated in FIG. 7A, in the third embodiment thedelivery optics 73 comprises a plurality of LED's 277 for providing thecolored illumination to the object 26. In this embodiment, a red laseris used as the illumination source for the main optics when creating the3D entity. As such, this illumination means is also used to obtain thered monochromatic image for the creation of the 2D color image. Thus,the LED's 277 are each capable of providing an illumination radiation ineither green or blue, as controlled via the tri-color sequence generator74. The LED's 277 are located on the outer side of slanted face 94, andthus the LED's do not interfere with the other optical operations of thedevice 100 in particular including the transmission of the illuminatingradiation for the confocal focusing operations, or in the transmissionof reflected light from the object 26 to provide the 3D entity or the 2Dcolor entity. The slanted face 94 comprises a dichroic coating 278 onthe outer side thereof, which has relatively high reflectivity and lowtransmission properties with respect to red light, while havingsubstantially high transmission characteristics for blue light and greenlight, as illustrated in FIG. 7B. Thus, as illustrated in FIG. 7A, lightfrom each of the blue or green LED's 277 is transmitted, in turn,through the dichroic coating to interface 79 and thence to the object26, as controlled by the generator 74. At the same time the dichroiccoating permits internal reflection of the red radiation from the mainoptics 41 to the interface 79 and object 26, and thus allows the 3D scanto be completed, as well as allowing the red monochromatic image of theobject 26 to be taken by the device 100. Optionally, rather thanemploying blue and green LED's, tricolor LED's may be used, and properlysynchronized to illuminate with either green or blue light as controlledby generator 74. Alternatively, the illumination source for the mainoptics 41 may be a green or blue laser, in which case the LED's are eachcapable of providing illumination in the remaining two colors, and insuch cases the dichroic coating is adapted for allowing transmission ofthese remaining two colors while providing substantially high reflectionfor the illuminating laser of the main optics, in a similar manner tothat described above for the red laser, mutatis mutandis.

In a fourth embodiment of the device 100, and referring to FIG. 8,tri-color illumination is provided within the main focal optics 42, inparticular at the confocal system aperture stop, and facing theobjective lens of the system. An advantage provided by this form ofillumination is that the tri-color illumination illuminates the object26 through the downstream objective lens 142 in nearly collimated light,and thus the object illumination is highly uniform. The tri-color lightsources 377 may be mounted statically on the physical aperture stop atthe aperture stop plane 150, or alternatively they may be mounted on aretracting aperture stop, which also serves to stop down the systemaperture in preview mode. In this embodiment, by placing the tri-colorlight sources 377 at the aperture stop plane, wherein the light beamfrom the illumination source 31 narrows to a minimum within the mainoptics 41, the external dimensions of the device 100 may still remainrelatively compact.

Referring to FIG. 9, the tri-color light sources 377 may comprise, forexample, a plurality of tri-color LED's 385 mounted onto a bracket 380.The bracket 380 is typically annular, having a central aperture to allowillumination light from the illuminating unit 31 to pass therethroughand to the object 26, and to allow light coming from the object 26 topass therethrough and to the detection optics 60, without being affectedby the bracket 380. At the same time, the bracket 380 positions theLED's in the required location upstream of objective lens 166. The LED'sare arranged in a spaced radial and circumferential manner asillustrated in FIG. 9 to provide the most uniform illumination of theobject 26 possible with this arrangement. Typically, a red laser is usedas the illumination source 31 for the main optics 41 when creating the3D entity. As such, and as in other embodiments, this illumination meansis also used to obtain the red monochromatic image for the creation ofthe 2D color image. Thus, the LED's 385 are each capable of providing anillumination radiation in either green or blue, as controlled via thetri-color sequence generator 74. Alternatively, the illumination sourcefor the main optics 41 may be a green or blue laser, in which case theLED's 385 are each capable of providing illumination in the remainingtwo colors, in a similar manner to that described above for the redlaser, mutatis mutandis. Optionally, rather than employing blue andgreen LED's, tricolor LED's may be used, and properly synchronized toilluminate with either green or blue light as controlled by generator74. Further optionally, the LED's 385 may be used to provide,sequentially, all the required colored illuminations, typically red,green and blue. Alternatively, the LED's 385 each provide illuminationin one of at least three colors. Thus, some of the LED's 385 provide ablue illumination, while other LED's 385 provide green illumination,while yet other LED's 385 provide red illumination.

The device 100 according to a variation of the fourth embodiment isfurther adapted for providing improved precision of the color dataobtained therewith. In this connection, the device 100 according to thisvariation of the fourth embodiment is adapted such that the tri-colorlight sources 377 each illuminate the object 26 with as wide a depth offield as possible, i.e., at a low numerical aperture. Thus, each set oflight sources 377 of the same color, for example blue, illuminates aparticular depth of the object 26 in the z-direction while substantiallyin focus. In contrast, the numerical aperture of the confocal systemitself is relatively high to maximize accuracy of the depthmeasurements, and thus provides a relatively narrower depth of field.

Advantageously, the optical system downstream of the light sources 377,in this embodiment the objective lens 166, is chromatic, and inparticular maximizes the chromatic dispersion therethrough.Alternatively or additionally, a chromatic dispersion element, forexample an optically refractive block of suitable refractive index, maybe provided along the optical path between the light sources 377 and theobject 26. Thus, each one of the different-colored light sources 377illuminates a different portion of the object 26 along the z-direction.The light sources 377 providing the blue illumination illuminate infocus a portion of the object 26 closest to the device 100, and thelight sources 377 providing the red illumination illuminate in focus aportion of the object 26 furthest from the device 100. At the same time,the light sources 377 providing the green illumination illuminate infocus a portion of the object 26 intermediate the blue and red portions,and a non-illuminated gap may exists between the red and green, andbetween the green and blue illuminated portions, the depth of these gapsdepending on the dispersion characteristics of the downstream optics.Advantageously, the light sources 377 are also adapted for providingillumination in colors intermediate in wavelengths such as to illuminatethe aforesaid gaps in focus. Thus, the LED's 385 may be adapted forproviding both such additional colored illumination, or some of theLED's 385 may be adapted to provide colored illumination at a firstintermediate wavelength, while another set of LED's 385 may be adaptedto provide colored illumination at a second intermediate wavelength. Forexample, the first intermediate wavelength provides an illumination inaqua, and thus illuminates in focus at least a part of the gaps betweenthe blue and green illuminated focused zones of the object 26, while thesecond intermediate wavelength provides an illumination in amber, andthus illuminates in focus at least a part the gaps between the green andred illuminated focused zones. Of course, additional light sources maybe used to provide further intermediate wavelengths and thus providefurther depth cover illumination, in focus, of the object.

While the device 100 is used as a viewfinder, typically prior to takinga depth and color scan of the object 26, the above arrangement using atleast five different colored illuminations at a low numerical aperture,enables a much clearer and focused real-time color image of the object26 to be obtained. Thus when in operation in viewfinder mode (also knownas “aiming mode”, prior to the 3D scan event, while the dentalpractitioner is in the process of aiming the scanner onto the targetdental surface, for example) the device 100 according to this variationof the fourth embodiment repeatedly illuminates the object 26 in cycles,wherein in each cycle the object 26 is separately illuminated in each ofthe five colors blue, aqua, green, amber, red, in quick succession, andeach time a monochromatic image is obtained by the monochromatic imagesensor in 60. Each set of five monochromatic images is then analysed toprovide a composite color image, and this image is then displayed insubstantially real time in the viewfinder display window in the controlsoftware, so that the succession of such composite images gives theappearance of a substantially real-time color video feed of the object26.

Each of the monochrome images in any particular set corresponds to aparticular illumination color or wavelength, and thus the zone(s) of theobject 26 within the depth of field corresponding to this illuminationwill be in focus, while the other parts of the object 26 will appear outof focus. Thus, each such image in the aforesaid set of images willcontain a portion which has high precision focused image of a part ofthe object, for the particular illumination wavelength.

In forming a composite image for each set of images, the images arecombined in such a way as to maximize the precision of the focused imageand corresponding color thereof. Thus, for example, suitable algorithmsmay be applied to each of the five images of a set to distinguishbetween the focused and unfocused the areas thereof. Such algorithms mayemploy, for example, techniques which apply FFT techniques to areas ofthe images, and which search for high frequency portions whichcorrespond to focused areas. In any case, such algorithms, as well assoftware and hardware to accomplish the same are well known in the art.Then, the focused areas of each of the five images are merged to providea monochrome composite substantially focused image of the object. Next,the images obtained using the red, green and blue illuminations arecombined and converted to a corresponding luminescence/chroma (Y/C)image, and techniques for doing so are well known in the art. Finally,the luminescence component of the luminescence/chroma (Y/C) image isreplaced with the aforesaid corresponding composite focus image, and theresulting new luminescence/chroma image is then transmitted to thedisplay in the viewfinder.

For each set of images, prior to combining the corresponding red, greenand blue images, these are preferably first scaled to compensate formagnification effects of the different wavelengths. Thus, the greenimage, and more so the blue image, needs to be scaled up to match thered image.

When the user is ready to take a depth and color scan of the object 26,having steered the device 100 into position with the aid of theviewfinder, the device 100 takes a depth scan in the z-direction asdescribed herein, and either before or after the same, but in quicksuccession one with the other, takes a color scan in a similar manner tothat described above for the viewfinder mode, mutatis mutandis.Subsequently, the color data and the depth data of the two scans can becombined to provide the full spatial and color data for the surfacescanned.

Advantageously, one or more color scans may also be taken during thedepth scan, and/or at the beginning and at the end of the depth scan. Inone mode of operation, the depth scan is obtained by displacing theobjective lends 166 along the z-direction in a continuous or steppedmotion. Multiple color scans can then be obtained by associating thecolor sources 377 with the objective lens, so that these are alsodisplaced along the z-direction. Accordingly, as the light sources 377are moved in the z-direction towards the object 26 during the depthscan, at each different z-position in which a set of images is taken(concurrently with or alternately with the depth scan), each one of thecolored illuminations—red, green, blue and intermediatewavelengths—illuminates a progressively deeper part of the object alongthe z-direction. Of course, in some cases it is possible that at thedownstream end of the depth scan the green and red illuminationscompletely overshoot the object 26, and the corresponding images may bediscarded or otherwise manipulated to provide a composite color image atthis station. Thus, a plurality of color images can be obtained, eachbased on a different z-position, so that each illumination wavelength isused to illuminate in focus a different part (depth) of the object 26.Advantageously, suitable algorithms may be used to form a compositecolor image of the set of color images associated with a particularz-scan of the object 26 to provide even more precise and accurate colorimage, than can then be combined with the depth data.

Alternatively, and referring to FIG. 10, the tri-color light sources 377may be replaced with a rotating filter illumination system 400. Thesystem 400 comprises a while light source 410, such as for example whitephosphorus InGaN LED's, and the light therefrom is focused onto anoptical fiber bundle 420 by means of condenser optics 430. Between thecondenser optics 430 and the fiber bundle 420 is provided a rotatingtri-color filter 450. As best seen in FIG. 10A, the filter 450 isdivided into three colored sections, comprising blue, green and redfilters on adjacent sectors therein. The fiber bundle 420 is flared atthe downstream end 470 to form the desired illumination pattern.Optionally, the downstream end 470 of the fibers may be mounted onto anannular bracket similar to bracket 380 illustrated in FIG. 9, at theapertures stop plane of the confocal optics. A suitable motor 460,typically a stepper motor for example, drives the rotating filter suchas to sequentially present each colored filter to the light passing fromthe condenser optics 430 to the fiber bundle 420, as synchronized withthe sequence generator 74 (FIG. 1) to enable the detection optics 60 tocapture images of the object 26 when selectively illuminated with eachof the three colors. Optionally, if a red, blue or green illuminatingradiation is used as the illumination source 31 for the main optics 41when creating the 3D entity, then the rotating filter 450 only requiresto comprise the remaining two colors, as discussed above for similarsituations regarding the LED's, mutatis mutandis.

A fifth embodiment of system 100 is substantially similar to the fourthembodiment as described herein, with the following difference, mutatismutandis. In the fifth embodiment, and referring to FIG. 11, polarizersare provided at two locations in order to increase the image contrast. Afirst polarizing element 161 is located just downstream of the lightsources 377 so as to polarize the light emitted from the light sources377. A second polarizing element 162 is located just upstream of theimage sensor of the detection optics 60, and is crossed with respect tothe first polarizing element 161. Further, a quarter waveplate 163 isprovided just upstream of the object 26, i.e. at the downstream end ofthe endoscope 46 (FIG. 4A). The first polarizing element 161 istypically annular, having a central aperture to allow illumination lightfrom the illuminating unit 31 to pass therethrough and to the object,and to allow light coming from the object 26 to pass therethrough and tothe detection optics 60, without being affected by the polarizingelement 161. However, light that is reflected from the object 26 returnsto the main confocal optics 42 in a crossed polarization state due tothe effect of the quarter waveplate 163, and thus reaches the detectionoptics 60 at substantially full intensity. However, any light reflectedfrom the objective lens 166 of the confocal optics 42 is reflected atthe same polarization state, and is therefore filtered out by thecrossed polarizing element 162. This arrangement serves as an effectivesignal to ghost enhancement system.

A sixth embodiment of the system 100 is substantially as described forthe fourth embodiment, with the following difference, mutatis mutandis.In the sixth embodiment, and referring to FIG. 12, the tri-color lightsources 377 are replaced with a rotating filter illumination system 500.The system 500 comprises a while light source 510, such as for examplewhite phosphorus InGaN LED's, and the light therefrom is focused onto amirror 520 by means of condenser optics 530. Between the condenseroptics 530 and the mirror 520 is provided a rotating tri-color filter550, which is similar to the filter 450 illustrated in FIG. 11, and thuscomprises three colored sections, comprising blue, green and red filterson adjacent sectors therein, and is actuated by motor 560. The opticalaxis OA of the confocal optics 41 is orthogonal to the optical axis OA′of the light source 510 and condenser optics 530. The mirror 520 ismounted between the aperture stop plane and the objective lens 166 ofthe confocal optics, and at an angle to the optical axis OA thereof andto the optical axis OA′ of the light source 510 and condenser optics530. The mirror 520 is typically annular, having a central aperturealigned with optical axis OA to allow illumination light from theilluminating unit 31 to pass therethrough and to the, object 26, and toallow light coming from the object 26 to pass therethrough and to thedetection optics 60, without being affected by the mirror 520. At thesame time, the mirror 520 has sufficient reflecting surface to reflectlight from the source 510 via objective lens 166 and to the object 26.Optionally, if a red, blue or green illuminating radiation is used asthe illumination source 31 for the main optics 41 when creating the 3Dentity, then the rotating filter 550 only requires the remaining twocolors, as discussed above for similar situations, mutatis mutandis.

According to a second technique for providing the aforesaid 2D colorimage, the object 26 is illuminated with a white light, and a color CCDis used for receiving the light reflected from the object 26. Thus, aseventh embodiment of the system 100 comprises a white lightillumination system 600, illustrated in FIG. 13. The system 600comprises a while light source 610, such as for example white phosphorusInGaN LED's, and the light therefrom is directed onto a flip mirror 620via a polarizing beam splitter 650 by means of condenser optics 630. Theoptical axis OA of the confocal optics 41 is orthogonal to the opticalaxis OA″ of the light source 610 and condenser optics 630. The mirror620 is mounted between the aperture stop plane 155 and the objectivelens 166 of the confocal optics, and at an angle to the optical axis OAthereof and to the optical axis OA″ of the light source 610 andcondenser optics 630.

The mirror 620 is adapted to flip away from optical axis OA when thedevice 100 is being used for obtaining the 3D entity E. This allowsillumination light from the illuminating unit 31 to pass therethroughand to the object 26, and to allow light coming from the object 26 topass therethrough and to the detection optics 60, without being affectedby the mirror 620. When it is desired to take a 2D color image, themirror 620 is flipped down to the position shown in FIG. 13. Polarizingbeam splitter 650 that polarizes white light from the source 610 andallows the same to pass therethrough and to mirror 620, and thence tothe object 26 via the confocal objective 166 and broadband quarter waveplate 163. Light that is reflected from the object 26 returns to themirror 620 in a crossed polarization state due to the effect of thequarter waveplate 163, and thus reaches the color CCD 660 (andassociated detection optics—not shown) at substantially full intensity.However, any light reflected from the objective lens 166 of the confocaloptics 42 is reflected at the same polarization state, and is thereforefiltered out by a crossed polarizing element 662 just upstream of theCCD 660. This arrangement serves as an effective signal to ghostenhancement system.

Alternatively, the CCD of the detection optics 60 is a color CCD and isalso used for the 2D scan. In such a case, flipping mirror 620 isreplaced with a fixed mirror having a central aperture similar to mirror520, having a central aperture, as described for the sixth embodiment,mutatis mutandis.

In the seventh embodiment, the image capture device 80 and processingsoftware 82 (FIG. 4b ) automatically provide a 2D color image comprisingan array of data points having location (X, Y) and color (C) informationfor each pixel of the image.

According to a third technique for providing the 2D color image, theobject is illuminated with a white light, and the light reflected fromthe object 26 is passed sequentially through one of three differentcolored filters such as red, green and blue. Each time a monochromaticimage corresponding to each color is captured via CCD 68 and the imagecapture device 80 (see FIGS. 4A, 4B). Suitable processing software 82combines the three images to provide a 2D color image comprising anarray of data points having location (X, Y) and color (C) informationfor each pixel of the image.

According to a fourth technique for providing the color image, the mainillumination source 31 of device 100 comprises suitable means forproviding the three different colored illuminations. In one embodiment,the illumination source 31 comprises three different lasers, each oneproviding an illumination radiation at a different desired color, redgreen or blue. In another embodiment, a suitable white lightillumination means is provided, coupled to a suitable rotating tri-colorfilter, similar to the filters described above, mutatis mutandis. Ineach case, suitable control means are provided, adapted to illuminatethe object 26 with each colored radiation in turn, and the 2D coloredimage is obtained in a similar fashion to that described above, mutatismutandis. The object is also illuminated with one of the coloredilluminations in order to provide the 3D surface topology data.

In each of the embodiments described herein, the illumination radiationthat is used for obtaining the 2D color image is injected into theoptical axis OA of the confocal optics 42 without affecting theoperation thereof or degrading the 3D image capture.

The endoscope 46, the illumination unit 31, the main optics 41, colorillumination 71 and tri-color sequence genetrator are preferablyincluded together in a unitary device, typically a hand-held device. Thedevice preferably includes also the detector optics 60, though thelatter may be connected to the remainder of the device via a suitableoptical link such as a fibre optics cable.

For all embodiments, the data representative of the surface topology andcolor, i.e., entity I, may be transmitted through an appropriate dataport, e.g. a modem 88 (FIG. 4B), through any communication network, e.g.telephone line 90, to a recipient (not shown) e.g. to an off-siteCAD/CAM apparatus (not shown).

By capturing, in this manner, an image from two or more angularlocations around the structure, e.g. in the case of a teeth segment fromthe buccal direction, from the lingual direction and optionally fromabove the teeth, an accurate color three-dimensional representation ofthe teeth segment may be reconstructed. This may allow a virtualreconstruction of the three-dimensional structure in a computerizedenvironment or a physical reconstruction in a CAD/CAM apparatus.

While the present device has been described in the context of aparticular embodiment of an optical scanner that uses confocal focusingtechniques for obtaining the 3D entity, the device may comprise anyother confocal focusing arrangement, for example as described in WO00/08415. In fact, any suitable means for providing 3D scanning can beused so long as the 3D scan and the color 2D scan correspondsubstantially to the same object or portion thereof being scanned, andthe same frames of references are maintained. Typically the scans areexecuted in relatively quick succession, and by the same or differentimage capturing means such as CCD's that are arranged such that thecolor 2D image substantially corresponds to the 3D entity. This enablescolor values at particular x, y coordinates of the 2D color image to be,matched to the same x, y coordinates of the 3D image which also have a zcoordinate.

While four main embodiments of the present invention are now describedhereinbelow, it may be appreciated that the method of the invention maybe used for a very wide variety of applications in which intra oralcavity data may be obtained for use in procedures associated with theoral cavity.

Referring to FIG. 14, in a first embodiment of the present invention,the method of the invention, generally designated 1100, is particularlyadapted for automatically differentiating one tissue from another tissuewithin the oral cavity. As with all embodiments of the invention, thefirst step 1110 is to obtain numerical entity I comprising a data setrepresenting the surface geometry and colour of at least the portion ofthe intraoral cavity that contains the area of interest, herein referredto as the target surface. The target surface typically comprises bothhard tissues, such as the teeth, and soft tissues which include at leastone of the gums, tongue, cheeks and lips. Each data point in thenumerical entity I comprises surface coordinates of the tissue,typically expressed as Cartesian (x, y, z) coordinates, plus a colourparameter relating to the colour c of the tissue at these surfacecoordinates. The colour value for c can be conveniently expressed incomponent form in terms of the red, green and blue components (RGB), andin particular in terms of a colour component coefficient. For example, ared coefficient ratio Rc, may be defined as the intensity of the redcomponent divided by the average intensity of the red, green and bluecomponents, and is useful in differentiating between the hard and softtissues. The entity I may be obtained, for example, using the devicedescribed herein with respect to FIGS. 1 to 13.

In the next step 1120, the value of the colour parameter c is analysedfor each data point in I, and compared with at least one colourcriterion, and typically with at least two colour ranges R₁, R₂. Theranges R₁, R₂ each represent the values of the colour parameter expectedfor one or another of the two tissues, such as the teeth and gums. Forexample, the colour range R₁ for the teeth will include values for ctypically associated with the hard tissues including the teeth,comprising all appropriate shades appropriate for enamel, dentine, pulpand other parts of teeth. Similarly, the colour range R₂ for the softtissues will include values for c typically associated with gums,cheeks, lips and the tongue including all appropriate shades of pink andred associated with these tissues, including when the tissues are atleast partially drained of blood, as may happen, for example when thetissues are anaesthetised. In some cases it may be appropriate tocompare the value of the colour parameter c with a specific value, forexample a single (R, G, B) or Rc value, rather than a range of values.

Table I lists typical RGB values measured for gums, lips and teeth of aparticular patient. As may be seen from Table I, values of Rcsignificantly greater than unity indicate that the red component isdominant, which is the case for soft tissues in general. Hard tissues,on the other hand, have a more even distribution of colour components,resulting in an Rc value very close to unity.

TABLE I Intraoral RGB Values measured for a Patient Red Green Blue Rc =3*R/ Tissue component component component (R + G + B) Gum 1 76 28 28 1.6Gum 2 119 79 80 1.28 Lower lip 165 120 130 1.2 Premolar 168 175 167 0.99Incisor 172 174 170 1.0

Thus, an exemplary range for R₁ may be from about 0.9 to about 1.1,while an exemplary range for R₂ may be from less than about 1.2 to amaximum of 3.0.

The ranges R₁, R₂ should preferably be sufficiently spaced one from theother and not overlap to facilitate distinction between the differenttissues and to prevent ambiguity. At the same time, each range shouldinclude all possible variations of colour that may be expected for eachof the tissues involved.

The actual values of the ranges R₁, R₂ may vary between individuals. Forexample, some individuals may have yellowing teeth and pale softtissues, while others may have milky white teeth and reddish complexion.Optionally, it is possible to pre-calibrate the ranges R₁, R₂, forexample by scanning an area that is purely soft tissue, and anotherdental area that is purely hard tissue, and using the colour values forthe two scans as datum values for the two ranges R₁, R₂.

Optionally, each tissue type may also be associated with more than onerange of colour values, and thus each one of R₁, R₂ may comprise a setof ranges. For example, R₂ may actually include four separate ranges,each relating to the colour variations of one of the gums, cheeks, lipsand tongue. Also, R₁ may include a number of separate ranges, one rangerepresentative of the variations in the colour of natural teeth, whilethe other ranges relate to colours associated with prostheses and/orwith fillings, particularly when made from materials which do not give anatural appearance to the prostheses, for example gold crowns or amalgamfillings.

In the next step 1130, the data points for entity I are sorted into atleast two sub-data sets, I₂, according to the colour criteria, that is,for example, whether the value of the colour parameter of each datapoint is within R₁ or R₂, respectively. Thus, I₁ will contain all datapoints in which the colour parameter thereof corresponds to the colourof teeth, and thus should comprise the coordinates relating to thesurfaces of teeth only within the original entity I (optionallyincluding prostheses and fillings). Similarly, I₂ should contain alldata points in which the colour parameter thereof corresponds to thecolour of the soft tissues, and thus should comprise the coordinatesrelating to the soft tissues only within the original entity I.

In a modification to this embodiment, it may only be necessary ordesired to identify one tissue, such as the teeth for example, anddisregard all data not relating to this tissue. In such cases, it isonly necessary to compare the value of the colour component of each datapoint to a single colour criterion, such as for example a predeterminedrange R₁ relating to the teeth only, and then separate out these datapoints from the entity I to provide an entity I₁ that comprises datarelating only to the teeth. Of course, it may also be desired to includein this data set artificial teeth and also fillings that do not have anatural colour, and thus the range R₁ may optionally include theappropriate values for the colour parameter relating thereto.

Once the original entity I has been separated into two entities, orwherein an entity I₁ has been created from the original entity Icomprising only the tissue of interest, the new entity may be furthermanipulated as desired. In step 1140, for example, when the new entityI₁ comprises only teeth-related data, each individual tooth may beidentified therein. In such a situation, the entity I₁ is furtherseparated out into a plurality of smaller entities each of which relatesto a separate tooth. Typically, the separation of I₁ into isautomatically effected using any suitable algorithm

In step 1150, after the data relating to the individual teeth has beenproperly sorted, further manipulation may be carried out for each of theindividual data sets of the entities I₁′, for example to simulate aparticular orthodontic treatment for the teeth.

This embodiment may also be applied to the identification of a finishline profile for a crown or bridge prosthesis.

The finish line may be regarded as the circumferential junction orshoulder between the upper prepared portion of the tooth and the lowerunprepared portion of the tooth. The finish line may be above or belowthe visible gum line, i.e. the exterior visible line of gingival tissuewhich circumferentially surrounds the tooth. Frequently, the finish lineis below the visible gum line and is uneven, i.e. the finish line variesin height along the circumferential direction and can rise or fall onthe order of several millimetres in the generally vertical direction.The finish line may even, in some cases, extend as far downwardly as theattachment line, i.e. the circumferential line defined by the neck ofthe tooth and its immediately adjacent gingival tissue below theaforementioned visible gum line. As with the finish line, the attachmentline is uneven and also typically varies several millimetres in heightalong the circumferential direction. The contour of the attachment linevaries from tooth to tooth, as well as from patient to patient, and isnot readily visible or accessible to the dentist because it is below thevisible gum line. In such cases, a retention ring or wire, made of anelastically deformable material, may be placed around the preparation toretract the gum tissue around the preparation. The ring thus in manycases exposes at least part of the emerging profile—the surface of thetooth between the finish line and the gum.

The ring thus adopts a profile which may often be substantially similarto that of the finish line. By having the ring coloured sufficientlydifferently to the colour of the teeth or soft tissues, say in blue, itis relatively straightforward to separate out from the entity I all datapoints having a colour component with a value in a specific rangecorresponding to the colour of the ring. Identification of the ringitself provides a useful starting point for suitable algorithms that arethen applied to determine the location and geometry of the finish line.Such algorithms are known and typically attempt to identify featurescommonly found with finish lines such as for example, a discontinuity inthe slope of the tooth surface, or a mound-shaped projectioncorresponding to the preparation. Moreover, separation of the hardtissue from the soft tissue results in a smaller data base that needs tobe analysed to identify the finish line. In particular, when the dataset has been separated into entities I₁, then only the specific entityI₁ corresponding to the ring needs to the analysed for the finish line,as this entity corresponds to the preparation.

In all variations of this embodiment, the comparison of value for thecolour parameter c with an appropriate range, and the sorting of datapoints into one or more data sets according to this comparison can beexecuted with any suitable computer with the aid of a suitablyconstructed program. The manipulation of the entities at each stage withrespect to the computer may be manual, interactive, or partially orfully automated.

In the second embodiment of the present invention, and referring to FIG.15, the method, generally designated 1200, finds particular use foraccurately stitching data sets of overlapping areas scanned within theintra-oral cavity. In the first step 1210, a suitable scanner, asdescribed herein, for example, may be used for obtaining high resolutionnumerical sub-entities (IS₁, IS₂, . . . IS_(n)) representative of thesurface topography and colour of zones (A₁, A₂, . . . A_(n)) within theintra-oral cavity. The zones that are scanned should together fully spanthe target zone of interest. Some of these zones may overlap with allthe other zones, while other zones may only overlap with one or twoother zones, and thus theoretically the corresponding entities shouldhave a portion of their data identical to one another within the overlapzone. However, since the scanner itself moves in orientation andposition from scan to scan, the coordinates relating to these overlapzones will usually vary between adjacent scans, even though they relateto the same spatial feature. Nevertheless, by identifying the overlapzone within the entities, the spatial relationship between the entitiesmay be established, and the various entities stitched together orcombined to form a global numerical entity that comprises the fullgeometry and colour representation of the target zone. The larger theoverlap zone, the greater the accuracy with which the different zonesmay be synchronized together spatially.

In prior art methods, the overlap zones may be identified by numericallytransforming the coordinates of an entity associated with one zone—by aseries of translations and rotations—and in each case comparing the dataset with the data set of another entity. This process is repeated untilat least a portion of the data from the first entity coincides with atleast a portion of the data from another entity. At this point, the datasets comprising the two sub-entities can be combined by adding both setsof coordinates, and discarding every data point that is repeated.However, some ambiguity may occur when using such a technique if a partof the intra-oral cavity (corresponding to part of the overlappingscanned data in some of the entities) moves in relation to other parts.For example, between one scan and another scan, part of the cheek maymove relative to the teeth. It is then problematic to construct acomposite entity comprising both scans since at least a part of thetissues (in this example the cheek) will be associated with twodifferent data portions representative of the relative movement betweenscans.

In the second embodiment of the present invention, a method forstitching different data sets for the intraoral cavity is provided, inwhich the actual stitching technique is applied to data pointscorresponding to the hard tissues therein. Accordingly, in a second step1220 of the method, the hard tissues including the teeth, fillings andprostheses are differentiated from the soft tissues including the gums,cheeks, lips and tongue. Substantially the same method as describedabove for first embodiment of the invention above may be utilized toidentify the data, in each of the sub-entities (IS₁, IS₂, . . . IS_(n))that is associated with the hard tissues, mutatis mutandis. The data inthese entities not corresponding to the hard tissues may de discarded orsimply noted and set aside for future reference, thereby providingmodified entities (IS′₁, IS′₂, . . . IS′_(n)) comprising the data ofinterest relating to the hard tissues, for example.

In the next step 1230 of the method, the modified entities (IS′₁, IS′₂,. . . IS′_(n)), are then manipulated in a similar manner in which theoriginal entities (I₁, I₂, . . . I_(n)) are manipulated in the priorart, mutatis mutandis, to register and then stitch the various modifiedentities (IS′₁, IS′₂, . . . IS′_(n)) together to provide a compositeentity I′ that comprises the data points corresponding at least to thehard tissues.

As an optional step, the data referring to the soft tissues may then beadded to the composite entity I′ as follows. Referring to the softtissue data corresponding to each scan as entities (IS″₁, IS″₂, . . .IS″_(n)), each one of these entities is first manipulated in preciselythe same manner as the corresponding entity of the group of modifiedentities (IS′₁, was manipulated in order to stitch the latter togetherinto I′. After this, the coordinates of each pair of entities within thegroup (IS″₁, IS″₂, . . . IS″_(n)) are compared in turn. Each pair ofentities within the group (IS″₁, IS″₂, . . . IS″_(n)) is checked todetermine whether there exist some data points in one entity having twocoordinates, say (x, y) identical to corresponding data points in theother entity, but in which the (z) coordinates are different. All suchdata points in either one or the other entity are then disregarded. Inthis manner, a composite entity I″ can be constructed for the softtissues, which can then be added to, if desired to the composite entityI′ of the hard tissues previously created.

Typically, course stitching of the original sub entities (IS₁, IS₂, . .. IS_(n)) is first carried out, and when the approximate relationshipsbetween the sub entities is known, a next step is performed, comprisingfine stitching the corresponding separated hard-tissue sub entities(IS′₁, IS′₂, . . . IS′_(n)).

In the third embodiment of the present invention, and referring to FIG.16, the method, generally designated 1300, is used for automaticallydefining the finish line of a preparation. As described above, thefinish line may be regarded as the circumferential junction or shoulderbetween the upper prepared portion of the tooth and the lower unpreparedportion of the tooth. The finish line may be above or below the visiblegum line, i.e. the exterior visible line of gingival tissue whichcircumferentially surrounds the tooth. In cases where the finish lineprotrudes from the gum, the complete circumferential extent of thefinish line is visible and appears as a shade of white. This may becontrasted with the pink/red colour of the surrounding gums, and at thesame time the finish line is assumed to be in close proximity to thegums. In such a case, the method of the invention according to the thirdembodiment is carried out as follows.

As with other embodiments, the first step 1310 is to provide a numericalentity I that describes the target area—in this case the part of theintraoral cavity that comprises the finish line—geometrically and withrespect to colour. Preferably, the target area is confined to the toothhaving the preparation, and possibly the adjacent teeth. Then, in step1320, an algorithm is applied to every pair of spatially adjacent datapoints, for example, wherein the value of the colour parameter c of eachof these points are compared one with the other, or with respect to somecolour criteria. When it is determined that the difference in colourvalues is greater than a predetermined threshold, it is then assumedthat the pair of data points are on opposite sides of a boundary betweentwo tissues of different colour, in particular between the edge of thetooth and the gum. Alternatively, the value of parameter c is comparedwith two predetermined ranges, R₁, R₂, wherein each range corresponds toa colour associated with one or the other of teeth and gums. Then,wherever there is a pair of adjacent data points in which one data pointhas a value for parameter c within R₁ and the other data point has avalue for parameter c within R₂, once again these two data points areconsidered to be on either side of the boundary between a tooth and thegum. The process is repeated for each adjacent pair of points in theentity I, thereby providing in step 1330 another numerical entity I_(FL)representative of the gum line and comprising topographic (as well ascolour) information relating thereto.

Identification of the gum line itself provides a useful starting pointfor suitable algorithms that are then applied in step 1340 to determinethe location and geometry of the finish line. Such algorithms typicallyattempt to identify features commonly found with finish lines, as hasbeen described herein for the first embodiment. Thus, the algorithm isapplied to the entity I, but staring with the surface data thereofcorresponding to entity I_(FL).

In cases where the finish line is partially or fully located below thegum, a suitable ring may be placed between the neck of the toothpreparation and the gum such as to retract the latter and expose thefinish line. The method according to the third embodiment may then beapplied with a modification in that the boundary between the toothmaterial and the ring material is searched for, in a similar manner tothat described regarding the boundary between the tooth material andgum, mutatis mutandis, which provides a starting point for algorithmsthat are then applied to identify the finish line.

Preferably, where the entity I is viewable as a two dimensional colourimage on a screen, the method optionally further comprises the step ofdisplaying on such an image of the entity I the finish line entityI_(FL), preferably in a colour having high contrast with respect to I,to enable the practitioner to check the result.

The method according to this embodiment may be modified to separate thesoft tissues from the hard tissues, once the demarcation line betweenthe two tissues is known, as determined above.

In the fourth embodiment of the invention, and referring to FIG. 17, themethod, generally designated 1400, is particularly applied to theshading of a prosthesis. As with the other embodiments of the invention,the first step 1410 involves the creation of a numerical entity I of theregion of interest within the oral cavity. Referring to FIG. 18, in thiscase, the region of interest R includes the preparation area P of theintraoral cavity in which the prosthesis is to be mounted, includingadjacent teeth thereto, A, B as well as the teeth C, D, E oppositethereto on the opposing jaw. The region of interest R may optionally oralternatively include at least one tooth or teeth on the other side ofthe same jaw corresponding to the tooth or teeth that the prosthesis isdesigned to replace. The desired data in this embodiment that it isdesired to create from the entity I relates to shading information forthe prosthesis such as to provide a natural-looking appearance theretothat is compatible with the other teeth of the patient. The shadingcharacteristics of the prosthesis according to the invention are to bebased on the shading characteristics of at least one tooth of thepatient, and thus share the same colour, variations in hue and otherfeatures such as reflective qualities and translucency, that may befound on one or more teeth of the patient, according to any desired setof rules or algorithm.

In the next steps 1420, 1430, 1440, the entity I is manipulated such asto extract the data corresponding to the teeth only, and to separatethis data into a set of discrete entities each of which represents anindividual teeth, substantially as described regarding steps 1120, 1130and 1140, respectively, for the first embodiment herein, mutatismutandis.

Then, in step 1450, the decision is taken regarding which teeth are tobe considered for modelling the shading of the prosthesis thereon. Thisdecision may be made automatically, for example including only theadjacent teeth A, B, or the tooth D directly opposite the preparationarea P, or any other appropriate tooth or combination of teeth. Asuitable algorithm may be provided that recognizes which entity inI_(1′). corresponds to the area P, for example by determining the heightof each tooth entity, and working out the entity having the shortestheight. The height of each tooth may be obtained from the correspondingentity I_(F) by suitable algorithms and routines, as known in the art.The spatial relationship between the entity corresponding to the area Pand the entities corresponding to the other teeth can then be determinedin an automated manner by using simple geometrical rules.

Alternatively, the choice may be made manually, and this may be done,for example, by displaying the scanned and separated teeth entities on asuitable display, such as a computer screen for example, and thenmarking the desired tooth or teeth by means of a cursor or the like. Itmay be preferable to manually perform this selection, as it would beimportant not to include certain teeth with particular visual defects orirregularities, e.g. cavities, parts that require restorations, spotsthat need to be removed etc. that it is not wished to repeat in theprosthesis.

The data sets I_(P1′), corresponding to the chosen teeth are thenanalysed in turn, and in step 1460 a color map for the prosthesis isprovided based on the colour data provided from data base I_(P1)′.

Referring to FIG. 19, for each of the teeth chosen, the appropriateentities are each geometrically transformed to conform to apredetermined geometrical shape, for the sake of example illustrated inthis figure as a cylinder, to provide a transformed entity T. In thetransformation process, the surface geometry of I_(P1′) is transformedinto the surface geometry of T, and the values of the colour parametersc associated with every data point are correspondingly transferred.Effectively, then, the colour attributes of each individual entityI_(v), such as for example features z1, z2, z3, and the two differentlycoloured zones z4, z5 exemplified in FIG. 19, are thus mapped onto theentity T into areas Z1, Z2, Z3, Z4 and Z5 respectively.

When only one tooth is chosen for basing the shading of the prosthesison, i.e., corresponding to only a single entity I_(P1′), the transformedentity T thus obtained is then transformed again to assume the shape ofthe prosthesis, providing a prosthesis entity X, effectively mapping allthe colour features of the entity I_(P1′) thereto. The shape of theprosthesis is previously determined by any suitable means, and this doesnot form part of the present invention. The entity X thus comprises thesurface geometry of the prosthesis, and the shading information withrespect to this geometry, including features x1-x5 transferred fromfeatures Z1-Z5 respectively, that will provide a similar appearance tothat of the tooth on which the colour was modelled. In such a case, theintermediate step of transforming I_(P1′) to T may be dispensed with,and thus the entity I_(P1′) may be transformed directly into the form ofthe crown prosthesis, thereby providing the values for the colour cthereof.

Nevertheless, the inclusion of intermediate entity T may be useful.

Optionally, the tooth may be divided into three general zones: agingival zone close to the gums, an intermediate body zone, and anincisal zone comprising the cusp of the tooth. The colours can be mappedinto each of the zones independently, and then smoothed out between thezones to avoid sharp discontinuities in colour.

When a number of teeth are chosen to serve as the basis for shading theprosthesis, the entities I_(1′) corresponding to each of these teeth istransformed to a corresponding entity T′. Then, the colour data of theentities T′ are combined to provide a composite entity T of the sameshape but having composite shading information obtained from all theentities I_(1′). For example the colour value at every geometrical pointfor all the entities T′ could be averaged. Alternatively, a weightedaverage of the colours could be provided, wherein more weight is givento teeth that are closest in location and/or function than to otherteeth. Again, when such a combination of the colour information iseffected, it is important to ensure that the various entities T′ arealigned with respect to the individual interproximal, buccal and lingualsides. The composite entity T is then transformed geometrically toconform to the shape of the prosthesis to provide entity X as describedbefore, but wherein the composite colour values for parameter c are nowtransferred to the geometrical shape of the entity.

In the above example, the prosthesis has been exemplified as a crown.Nevertheless, the method may be applied to a bridge prosthesis, filing,restoration, or tooth transplant in a similar manner to that described,mutatis mutandis.

While the above embodiments have been described as operations carriedout on discrete data points, it is clear that the method of theinvention is applicable to similar operations being carried out on newdata points suitably interpolated with respect to the original datapoints, mutatis mutandis. Furthermore, it is also possible to carry outthe method of the invention when the numerical entity is structured in adifferent manner to that described herein, mutatis mutandis. Forexample, rather than being described as a series of discrete data pointson a surface, the surface of the intra-oral cavity could be described asa series of segments, by suitable formulae, or by any other geometricmodelling method.

FIG. 20 is a block diagram of a system 1500 according to an embodimentof the present invention. System 1500 presents a software/hardwareutility connectable to a virtual treatment system 1510, for example.System 1500 comprises such main elements as follows:

-   -   an input utility 1520 for receiving data indicative of three        dimensional colour data of the intraoral cavity, obtained        directly using scanner 1525 (for example the device 100        described herein in reference to FIGS. 1 to 13) or obtained        elsewhere and transmitted to the input facility via any        communication medium 1535, including for example the internet,        an intranet, and so on;    -   a memory utility 1530 for storing the data;    -   a processor utility 1540 for manipulating said first data set to        provide desired data therefrom, particularly using any algorithm        according to the present method; and    -   output utility 1550.

System 1500 can be connected to a display 1560 or a printer (not shown)for visually presenting the manipulated entities. System 1500 can bealso connectable to an additional utility such as a virtual treatmentsystem 1510.

In another aspect of the present invention, a computer readable mediumis provided that embodies in a tangible manner a program executable forproviding data useful in procedures associated with the oral cavity. Thecomputer readable medium comprises:

-   -   (a) a first set of data representative of the three dimensional        surface geometry and colour of at least part of the intra oral        cavity;    -   (b) means for manipulating said first data set to provide        desired data therefrom.

The medium may comprise, for example, optical discs, magnetic discs,magnetic tapes, and so on.

The embodiments illustrated herein are particularly useful fordetermining the three-dimensional structure of a teeth segment,particularly a teeth segment where at least one tooth or portion oftooth is missing for the purpose of generating data of such a segmentfor subsequent use in design or manufacture of a prosthesis of themissing at least one tooth or portion, e.g. a crown, or a bridge, or adental restoration or a filing. It should however be noted, that theinvention is not limited to this embodiment, and applies, mutatismutandis, also to a variety of other applications of imaging ofthree-dimensional structure of objects, e.g. for the recordal orarcheological objects, for imaging of a three-dimensional structure ofany of a variety of biological tissues, etc.

While there has been shown and disclosed exemplary embodiments inaccordance with the invention, it will be appreciated that many changesmay be made therein without departing from the spirit of the invention.

In the method claims that follow, alphabetic characters and Romannumerals used to designate claim steps are provided for convenience onlyand do not imply any particular order of performing the steps.

Finally, it should be noted that the word “comprising” as usedthroughout the appended claims is to be interpreted to mean “includingbut not limited to”.

While preferred embodiments of the present invention have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. Numerousvariations, changes, and substitutions will now occur to those skilledin the art without departing from the invention. It should be understoodthat various alternatives to the embodiments of the invention describedherein may be employed in practicing the invention. It is intended thatthe following claims define the scope of the invention and that methodsand structures within the scope of these claims and their equivalents becovered thereby.

What is claimed is:
 1. A system for generating a 3D model of anintraoral structure portion, the system comprising: a handheld imagingdevice comprising a probe, focusing optics configured to scan a focalplane of the handheld imaging device over a range of depths, anilluminator, and an image sensor configured to capture image data overthe range of depths of the intraoral structure, wherein the image dataincludes depth data captured in response to illuminating the intraoralstructure portion with light of a first wavelength and image datacaptured in response to illuminating the intraoral structure portionwith light of a second wavelength; and one or more processors operablycoupled to the hand-held imaging device, the one or more processorsconfigured to cause the system to: receive at least two numericalentities representative of the intraoral structure and comprising thedepth data and the image data; differentiate between depth datarepresenting hard tissue of the intraoral structure and depth datarepresenting soft tissue of the intraoral structure based on the imagedata; stitch the at least two numerical entities together based on aweighted registration of the depth data, the weight being based on thedifferentiation between the depth data representing hard tissue anddepth data representing soft tissue.
 2. The system of claim 1, whereinthe at least two numerical entities comprise surface data of theintraoral portion.
 3. The system of claim 1, wherein the illuminatoremits light of the first wavelength and first image data comprises lightof the first wavelength returning from the intraoral structure.
 4. Thesystem of claim 3, wherein the light of the first wavelength returningfrom the intraoral structure is reflected from the intraoral structure.5. The system of claim 1, wherein second image data includes color datapoints.
 6. The system of claim 5, wherein the differentiation betweenthe hard tissue and the soft tissue is based on the color data.
 7. Thesystem of claim 6, wherein the weighted registration comprises onlyregistration of the depth data representing the hard tissue.
 8. Thesystem of claim 6, wherein the differentiation between depth datacomprises comparing color data points within the image data to at leastone color criterion and the one or more processors are furtherconfigured to cause the system to associate each of a plurality ofcorresponding depth data points with either hard tissue or soft tissuebased on the comparison.
 9. The system of claim 6, wherein the one ormore processors are further configured to cause the system to determinea color coefficient for each data point in the image data.
 10. Thesystem of claim 9, wherein the differentiation between depth datacomprises comparing the color coefficient for each data point in theimage data to at least one color criterion and the one or moreprocessors are further configured to cause the system to associate eachof a plurality of corresponding depth data points with either hardtissue or soft tissue based on the comparison.
 11. The system of claim1, wherein differentiation between hard tissue and soft tissue alsoincludes differentiation between types of soft tissue.
 12. A method forgenerating a 3D model of an intraoral structure, the method comprising:illuminating the intraoral structure with a first light source;capturing a plurality of depth image data sets during the illuminationwith first light source; illuminating the intraoral structure with asecond light source; capturing a plurality of two-dimensional image dataduring the illumination with the second light source; differentiatingbetween hard tissue in the plurality of depth image data sets and softtissue in the plurality of depth image data sets based on the pluralityof two-dimensional image data; and stitching the plurality of depthimage data sets together based on the differentiation between the hardtissue and the soft tissue.
 13. The method of claim 12, wherein thedepth data comprise surface data of the intraoral structure.
 14. Themethod of claim 12, wherein the depth image data comprises light of thefirst light source returning from the intraoral structure.
 16. Themethod of claim 15, wherein the two-dimensional image data compriseslight of a first wavelength from the intraoral structure.
 17. The methodof claim 12, stitching the plurality of depth image data sets comprisesa weighted registration of data points of the plurality of depth imagedata sets, the weighted registration based on the differentiationbetween the hard tissue and the soft tissue.
 18. The method of claim 17,wherein the weighted registration comprises only registration of thedepth data representing the hard tissue.
 19. A system for determiningsurface characteristics of an intraoral structure, the systemcomprising: a hand-held imaging device comprising a probe, focusingoptics configured to scan a focal plane of the handheld imaging deviceover a range of depths, a first light source, a second light source, andan image sensor configured to capture depth image data over the range ofdepths in response to illuminating the intraoral structure portion withthe first light source and capture two-dimensional image data inresponse to illuminating the intraoral structure portion with the secondlight source; and one or more processors operably coupled to the imagingdevice, the one or more processors configured to cause the system to:receive a plurality of depth image data sets comprising light of a firstwavelength; receive a plurality of two-dimensional image data comprisinglight of a second wavelength; differentiate between hard tissue in theplurality of depth image data sets and soft tissue in the plurality ofdepth image data sets based on the plurality of two-dimensional imagedata; and stitch the plurality of depth image data sets together basedon the differentiation between the hard tissue and the soft tissue. 20.The system of claim 19, wherein the three-dimensional numerical entitycomprises a plurality of data points, each data point comprisingthree-dimensional surface coordinate data and two-dimensional dataassociated therewith.
 21. The system of claim 19, wherein the imagesensor is a color image sensor.
 22. The system of claim 19, wherein thedifferentiation between hard tissue in the plurality of depth image datasets and soft tissue in the plurality of depth image data sets based onthe plurality of two-dimensional image data comprises comparing colordata points within the plurality of two-dimensional image data to atleast one color criterion and the one or more processors are furtherconfigured to cause the system to associate each of a plurality ofcorresponding depth data points with either hard tissue or soft tissuebased on the comparison.
 23. The system of claim 19, wherein thetwo-dimensional image data is color image data.
 24. The system of claim19, wherein the depth data is surface data of the intraoral portion. 25.The system of claim 19, wherein the illuminator emits light of at leastthe first wavelength and the plurality of depth image data sets compriselight of the first wavelength returning from the intraoral structure.26. The system of claim 25, wherein the light of the first wavelengthreturning from the intraoral structure is reflected from the intraoralstructure.
 27. The system of claim 19, wherein the one or moreprocessors are further configured to cause the system to determine acolor coefficient for each data point in the image data; and wherein thedifferentiation between depth data comprises comparing the colorcoefficient for each data point in the image data to at least one colorcriterion and the one or more processors are further configured to causethe system to associate each of a plurality of corresponding depth datapoints with either hard tissue or soft tissue based on the comparison.28. The system of claim 19, wherein the focusing optics, the first lightsource, the second light source, and the image sensor are configured to:sequentially illuminate the intraoral structure portion with each of aplurality of wavelengths of emitted light; and capture the plurality ofdepth image data sets and the plurality of two-dimensional image dataduring the sequential illumination of the intraoral structure, each ofthe plurality of plurality of depth image data sets and the plurality oftwo-dimensional image data being captured during emission of arespective one of the plurality of wavelengths.
 29. The system of claim28, wherein the plurality of wavelengths includes the first and secondwavelengths.
 30. The system of claim 29, wherein the image sensor is acolor image sensor.